[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:16 dist-test-slave-d0b5 krb5kdc[2534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:16 dist-test-slave-d0b5 krb5kdc[2534](info): set up 2 sockets
Jun 19 08:14:16 dist-test-slave-d0b5 krb5kdc[2534](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 08:14:17.154397 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:17.272357 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:44175
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:44175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:17.368086 2547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:17.368289 2547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:17.368312 2547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:17.369437 2547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:17.369472 2547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:17.369485 2547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:17.369498 2547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:17.369509 2547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:17.370913 2547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:44175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:44175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:17.371109 2547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:17.371284 2547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:17.376231 2553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.376264 2555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.376410 2547 server_base.cc:1034] running on GCE node
W20260619 08:14:17.376246 2552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.376801 2547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:17.377055 2547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:17.378201 2547 hybrid_clock.cc:648] HybridClock initialized: now 1781856857378193 us; error 35 us; skew 500 ppm
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:17.380733 2547 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:17.381105 2547 webserver.cc:460] Webserver started at http://127.2.119.254:42447/ using document root <none> and password file <none>
I20260619 08:14:17.381338 2547 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:17.381395 2547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:17.381505 2547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:17.382508 2547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "1694ef340a3b42fca05a868f5bdd4f42"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "0de6dd22502d164a769e0fcb5cc6c4cd"
server_key_iv: "c747cf61ae1b74b99156e0a54ab6c6ef"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.382815 2547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "1694ef340a3b42fca05a868f5bdd4f42"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "0de6dd22502d164a769e0fcb5cc6c4cd"
server_key_iv: "c747cf61ae1b74b99156e0a54ab6c6ef"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.384101 2547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:17.384771 2562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.384928 2547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:17.385013 2547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "1694ef340a3b42fca05a868f5bdd4f42"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "0de6dd22502d164a769e0fcb5cc6c4cd"
server_key_iv: "c747cf61ae1b74b99156e0a54ab6c6ef"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.385068 2547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:17.413522 2547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:17.415928 2547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:17.416051 2547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:17.419862 2547 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:44175
I20260619 08:14:17.419973 2614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:44175 every 8 connection(s)
I20260619 08:14:17.420277 2547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:17.421268 2615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:17.423614 2615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42: Bootstrap starting.
I20260619 08:14:17.424201 2615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:17.424479 2615 log.cc:824] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:17.425247 2615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42: No bootstrap required, opened a new log
I20260619 08:14:17.426889 2615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } }
I20260619 08:14:17.427053 2615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:17.427094 2615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1694ef340a3b42fca05a868f5bdd4f42, State: Initialized, Role: FOLLOWER
I20260619 08:14:17.427273 2615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } }
I20260619 08:14:17.427367 2615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:17.427409 2615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:17.427414 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 2547
I20260619 08:14:17.427459 2615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:17.427524 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:17.428009 2615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } }
I20260619 08:14:17.428124 2615 leader_election.cc:302] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1694ef340a3b42fca05a868f5bdd4f42; no voters:
I20260619 08:14:17.428200 2527 external_mini_cluster.cc:1428] Setting key 27ccf7087a073c605cb425e176eceee7
I20260619 08:14:17.428319 2615 leader_election.cc:288] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:17.428337 2618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:17.428489 2618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [term 1 LEADER]: Becoming Leader. State: Replica: 1694ef340a3b42fca05a868f5bdd4f42, State: Running, Role: LEADER
I20260619 08:14:17.428575 2618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } }
I20260619 08:14:17.428623 2615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:17.428895 2619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1694ef340a3b42fca05a868f5bdd4f42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } } }
I20260619 08:14:17.429061 2619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:17.429044 2620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1694ef340a3b42fca05a868f5bdd4f42. Latest consensus state: current_term: 1 leader_uuid: "1694ef340a3b42fca05a868f5bdd4f42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1694ef340a3b42fca05a868f5bdd4f42" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44175 } } }
I20260619 08:14:17.429198 2620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:17.429996 2628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:17.430536 2628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:17.432183 2628 catalog_manager.cc:1347] Generated new cluster ID: d2d42095eeff484198480e52fbfe5ec2
I20260619 08:14:17.432236 2628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:17.443930 2628 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:17.444324 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.429647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.2.119.254:44175)
0619 08:14:17.429838 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.429871 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.430503 (+ 632us) server_negotiation.cc:407] Connection header received
0619 08:14:17.432591 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.432596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.432640 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.432771 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.434469 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.435446 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.436581 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.437020 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.437145 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.440548 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:17.440556 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:17.440557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:17.440567 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:17.442355 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.442784 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.442787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.442787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.442871 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.443128 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.443128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.443129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.443340 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:17.443500 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:17.443910 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 08:14:17.444125 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":116,"threads_started":1}
I20260619 08:14:17.444790 2628 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:17.449223 2628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1694ef340a3b42fca05a868f5bdd4f42: Generated new TSK 0
I20260619 08:14:17.449407 2628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:17.475790 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:17.557621 2641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:17.557852 2641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:17.557894 2641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:17.559037 2641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:17.559101 2641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:17.559150 2641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:17.560711 2641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:17.560979 2641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:17.561201 2641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:17.566516 2649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.566519 2646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.566541 2647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.566670 2641 server_base.cc:1034] running on GCE node
I20260619 08:14:17.566987 2641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:17.567231 2641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:17.568396 2641 hybrid_clock.cc:648] HybridClock initialized: now 1781856857568381 us; error 27 us; skew 500 ppm
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:17.570960 2641 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:17.571403 2641 webserver.cc:460] Webserver started at http://127.2.119.193:33381/ using document root <none> and password file <none>
I20260619 08:14:17.571590 2641 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:17.571630 2641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:17.571717 2641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:17.572785 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "16a998e0fcbb4928a98a57da553487d8"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1411b9c6ac1a9186c7efc98c8458789f"
server_key_iv: "4e8d7fb0240f004734ddc0e684b2d204"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.573102 2641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "16a998e0fcbb4928a98a57da553487d8"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1411b9c6ac1a9186c7efc98c8458789f"
server_key_iv: "4e8d7fb0240f004734ddc0e684b2d204"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.574398 2641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:17.575037 2656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.575198 2641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:17.575285 2641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "16a998e0fcbb4928a98a57da553487d8"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1411b9c6ac1a9186c7efc98c8458789f"
server_key_iv: "4e8d7fb0240f004734ddc0e684b2d204"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.575356 2641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:17.595497 2641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:17.597481 2641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:17.597635 2641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:17.597862 2641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:17.598160 2641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:17.598192 2641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.598249 2641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:17.598277 2641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.603873 2641 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:41257
I20260619 08:14:17.603946 2769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:41257 every 8 connection(s)
I20260619 08:14:17.604262 2641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:17.610642 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 2641
I20260619 08:14:17.610739 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:17.610903 2527 external_mini_cluster.cc:1428] Setting key 3e3b93ec8630bbacedc5e3a6ae7252b5
I20260619 08:14:17.614313 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.605002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44175 (local address 127.2.119.193:45353)
0619 08:14:17.605157 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:17.605199 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:17.605910 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:17.606202 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:17.606206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:17.606826 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:17.607018 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.607022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.607788 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.607789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.608565 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.608567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.608991 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.608992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.609109 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.609680 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:17.609688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:17.611220 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:17.613003 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.613009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.613011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.613221 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.613456 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.613457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.613457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.613559 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.613832 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:17.613837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:17.613949 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 08:14:17.614036 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:14:17.614149 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260619 08:14:17.614313 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.605039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:45353 (local address 127.2.119.254:44175)
0619 08:14:17.605095 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.605097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.605289 (+ 192us) server_negotiation.cc:407] Connection header received
0619 08:14:17.606041 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.606043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.606061 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.606123 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.607157 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.607688 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.608679 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.608892 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.608975 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.611314 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:17.611316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:17.611316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:17.611324 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:17.612923 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.613320 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.613321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.613321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.613388 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.613623 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.613624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.613625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.613770 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:17.613827 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:17.614075 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 08:14:17.614173 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:17.614881 2770 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44175
I20260619 08:14:17.614986 2770 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:17.615182 2770 heartbeater.cc:511] Master 127.2.119.254:44175 requested a full tablet report, sending...
I20260619 08:14:17.615705 2579 ts_manager.cc:194] Registered new tserver with Master: 16a998e0fcbb4928a98a57da553487d8 (127.2.119.193:41257)
I20260619 08:14:17.616331 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:45353
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:17.637416 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:17.716627 2775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:17.716850 2775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:17.716889 2775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:17.718094 2775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:17.718156 2775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:17.718200 2775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:17.719637 2775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:17.719894 2775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:17.720109 2775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:17.725041 2783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.725095 2775 server_base.cc:1034] running on GCE node
W20260619 08:14:17.725067 2781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.725067 2780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.725471 2775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:17.725795 2775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:17.726945 2775 hybrid_clock.cc:648] HybridClock initialized: now 1781856857726932 us; error 31 us; skew 500 ppm
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:17.729640 2775 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:17.730065 2775 webserver.cc:460] Webserver started at http://127.2.119.194:45035/ using document root <none> and password file <none>
I20260619 08:14:17.730291 2775 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:17.730336 2775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:17.730473 2775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:17.731536 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "2c145dba788f43f7ab168f603920dcec"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "5e32be015169dd9c9700c1f35daa3d86"
server_key_iv: "9c1303d6c604cc76ac201307c084733a"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.731915 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "2c145dba788f43f7ab168f603920dcec"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "5e32be015169dd9c9700c1f35daa3d86"
server_key_iv: "9c1303d6c604cc76ac201307c084733a"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.733242 2775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:17.734041 2790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.734226 2775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:17.734318 2775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "2c145dba788f43f7ab168f603920dcec"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "5e32be015169dd9c9700c1f35daa3d86"
server_key_iv: "9c1303d6c604cc76ac201307c084733a"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.734401 2775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:17.757570 2775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:17.759505 2775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:17.759671 2775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:17.759922 2775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:17.760234 2775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:17.760286 2775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.760327 2775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:17.760354 2775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.766244 2775 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:36295
I20260619 08:14:17.766289 2903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:36295 every 8 connection(s)
I20260619 08:14:17.766597 2775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:17.772157 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 2775
I20260619 08:14:17.772259 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:17.772436 2527 external_mini_cluster.cc:1428] Setting key 7418942b7b43f7b6bd2aebd9778017ac
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:17.776389 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.767246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:40913 (local address 127.2.119.254:44175)
0619 08:14:17.767304 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.767306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.767523 (+ 217us) server_negotiation.cc:407] Connection header received
0619 08:14:17.768029 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.768030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.768041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.768100 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.769039 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.769648 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.770610 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.770811 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.770867 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.773341 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:17.773342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:17.773343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:17.773350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:17.775057 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.775445 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.775446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.775446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.775509 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.775723 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.775724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.775725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.775877 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:17.775946 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:17.776171 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 08:14:17.776259 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:14:17.776463 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.767271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44175 (local address 127.2.119.194:40913)
0619 08:14:17.767417 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:17.767453 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:17.767912 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:17.768178 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:17.768182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:17.768747 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:17.768943 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.768947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.769730 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.769730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.770537 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.770539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.770883 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.770884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.770982 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.771431 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:17.771438 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:17.773188 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:17.775152 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.775158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.775159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.775373 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.775573 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.775573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.775573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.775663 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.775939 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:17.775943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:17.776049 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 08:14:17.776128 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:14:17.776274 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260619 08:14:17.776911 2904 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44175
I20260619 08:14:17.777017 2904 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:17.777206 2904 heartbeater.cc:511] Master 127.2.119.254:44175 requested a full tablet report, sending...
I20260619 08:14:17.777614 2579 ts_manager.cc:194] Registered new tserver with Master: 2c145dba788f43f7ab168f603920dcec (127.2.119.194:36295)
I20260619 08:14:17.778092 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:40913
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:17.798238 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:33681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:17.880429 2909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:17.880684 2909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:17.880723 2909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:17.881907 2909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:17.881976 2909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:17.882020 2909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:17.883518 2909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:33681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44175
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:17.883797 2909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:17.884032 2909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:17.889231 2915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.889251 2914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:17.889256 2917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:17.889515 2909 server_base.cc:1034] running on GCE node
I20260619 08:14:17.889693 2909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:17.889950 2909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:17.891096 2909 hybrid_clock.cc:648] HybridClock initialized: now 1781856857891084 us; error 28 us; skew 500 ppm
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:17.893694 2909 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:17.894163 2909 webserver.cc:460] Webserver started at http://127.2.119.195:43493/ using document root <none> and password file <none>
I20260619 08:14:17.894419 2909 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:17.894488 2909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:17.894637 2909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:17.895784 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "2f12693bafdd4d9d80dfbc79ca75bdb0"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1a7d2c3023b63e340ff7786febf99730"
server_key_iv: "9ad5c6c91e52cac229f23f39f65fdab5"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.896152 2909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "2f12693bafdd4d9d80dfbc79ca75bdb0"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1a7d2c3023b63e340ff7786febf99730"
server_key_iv: "9ad5c6c91e52cac229f23f39f65fdab5"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.897569 2909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:17.898453 2924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.898617 2909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:17.898718 2909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "2f12693bafdd4d9d80dfbc79ca75bdb0"
format_stamp: "Formatted at 2026-06-19 08:14:17 on dist-test-slave-d0b5"
server_key: "1a7d2c3023b63e340ff7786febf99730"
server_key_iv: "9ad5c6c91e52cac229f23f39f65fdab5"
server_key_version: "encryptionkey@0"
I20260619 08:14:17.898787 2909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:17.925028 2909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:17.927045 2909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:17.927212 2909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:17.927444 2909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:17.927758 2909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:17.927793 2909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.927832 2909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:17.927870 2909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:17.934274 2909 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:45159
I20260619 08:14:17.934314 3037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:45159 every 8 connection(s)
I20260619 08:14:17.934684 2909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:17.942715 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 2909
I20260619 08:14:17.942798 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:17.942986 2527 external_mini_cluster.cc:1428] Setting key 3057061a099c141e25dd5245c1d3bd1a
I20260619 08:14:17.944188 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.935430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:33085 (local address 127.2.119.254:44175)
0619 08:14:17.935488 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.935490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.935716 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:14:17.936195 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.936196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.936207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.936266 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.937176 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.937746 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.938719 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.938917 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.938981 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.941131 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:17.941133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:17.941133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:17.941141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:17.942575 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.943095 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.943096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.943096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.943168 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.943471 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.943473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.943473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.943619 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:17.943704 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:17.944020 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 08:14:17.944101 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:14:17.944375 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.935405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44175 (local address 127.2.119.195:33085)
0619 08:14:17.935573 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:17.935617 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:17.936109 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:17.936320 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:17.936323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:17.936895 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:17.937092 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.937097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.937823 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.937824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.938641 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:17.938644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.939024 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.939025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:17.939152 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.939611 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:17.939620 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:17.941030 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:17.942675 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.942682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.942685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.942987 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.943237 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:17.943249 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:17.943250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:17.943392 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:17.943739 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:17.943744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:17.943880 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 08:14:17.943984 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 08:14:17.944140 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260619 08:14:17.944876 3038 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44175
I20260619 08:14:17.944970 3038 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:17.945138 3038 heartbeater.cc:511] Master 127.2.119.254:44175 requested a full tablet report, sending...
I20260619 08:14:17.945505 2579 ts_manager.cc:194] Registered new tserver with Master: 2f12693bafdd4d9d80dfbc79ca75bdb0 (127.2.119.195:45159)
I20260619 08:14:17.945868 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:33085
I20260619 08:14:17.955245 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:17.980082 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.971980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.2.119.193:41257)
0619 08:14:17.972122 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.972126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.972135 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:17.972159 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.972161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.972174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.972261 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.972844 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.973612 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.974402 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.974757 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.974837 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:17.977044 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:17.977053 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:17.977054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:17.977062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:17.978868 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.979234 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.979238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.979238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.979300 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:17.979516 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:17.979517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:17.979517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:17.979659 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:17.979819 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:17.979869 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 08:14:17.979953 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260619 08:14:17.980433 2684 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38238
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:14:17 dist-test-slave-d0b5 krb5kdc[2534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:18.003402 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:17.995950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.2.119.193:41257)
0619 08:14:17.996010 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:17.996012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:17.996076 (+ 64us) server_negotiation.cc:407] Connection header received
0619 08:14:17.996129 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:17.996130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:17.996144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:17.996204 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:17.996773 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.997270 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.998062 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:17.998306 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:17.998408 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:18.000755 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:18.000756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:18.000757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:18.000770 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:18.002320 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:18.002635 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:18.002636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:18.002636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:18.002706 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:18.002897 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:18.002897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:18.002898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:18.003014 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:18.003121 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:18.003192 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:14:18.003263 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:14:18.003914 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 2641
I20260619 08:14:18.008662 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 2775
I20260619 08:14:18.013923 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 2909
I20260619 08:14:18.018247 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 2547
2026-06-19T08:14:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1928 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:18 dist-test-slave-d0b5 krb5kdc[3052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:18 dist-test-slave-d0b5 krb5kdc[3052](info): set up 2 sockets
Jun 19 08:14:18 dist-test-slave-d0b5 krb5kdc[3052](info): commencing operation
krb5kdc: starting...
W20260619 08:14:19.082340 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:19.204533 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45843
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:19.284915 3065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:19.285141 3065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:19.285176 3065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:19.286381 3065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:19.286443 3065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:19.286468 3065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:19.286495 3065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:19.286525 3065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:19.287976 3065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:19.288231 3065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:19.288448 3065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:19.293197 3071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.293236 3070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.293315 3073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:19.293339 3065 server_base.cc:1034] running on GCE node
I20260619 08:14:19.293639 3065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:19.293941 3065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:19.295109 3065 hybrid_clock.cc:648] HybridClock initialized: now 1781856859295095 us; error 42 us; skew 500 ppm
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:19.297555 3065 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:19.298024 3065 webserver.cc:460] Webserver started at http://127.2.119.254:44353/ using document root <none> and password file <none>
I20260619 08:14:19.298245 3065 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:19.298310 3065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:19.298452 3065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:19.299513 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "887167dd5bf841128356e0dae8a3c50e"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "2e9800604349f69945d05571663c4a94"
server_key_iv: "e592f9b4a03e717dbe18838d4d2a2880"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.299875 3065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "887167dd5bf841128356e0dae8a3c50e"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "2e9800604349f69945d05571663c4a94"
server_key_iv: "e592f9b4a03e717dbe18838d4d2a2880"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.301201 3065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:19.301929 3080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.302099 3065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:19.302196 3065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "887167dd5bf841128356e0dae8a3c50e"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "2e9800604349f69945d05571663c4a94"
server_key_iv: "e592f9b4a03e717dbe18838d4d2a2880"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.302273 3065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:19.317623 3065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:19.319435 3065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:19.319594 3065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:19.323562 3065 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45843
I20260619 08:14:19.323601 3132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45843 every 8 connection(s)
I20260619 08:14:19.323961 3065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:19.324819 3133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:19.327066 3133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e: Bootstrap starting.
I20260619 08:14:19.327649 3133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:19.327955 3133 log.cc:824] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:19.328629 3133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e: No bootstrap required, opened a new log
I20260619 08:14:19.329042 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3065
I20260619 08:14:19.329134 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:19.329313 2527 external_mini_cluster.cc:1428] Setting key 04b22a4a6963dcb36ffa7f5b4c1660be
I20260619 08:14:19.330747 3133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } }
I20260619 08:14:19.330919 3133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:19.330958 3133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887167dd5bf841128356e0dae8a3c50e, State: Initialized, Role: FOLLOWER
I20260619 08:14:19.331097 3133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } }
I20260619 08:14:19.331183 3133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:19.331219 3133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:19.331270 3133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:19.332263 3133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } }
I20260619 08:14:19.332351 3133 leader_election.cc:302] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 887167dd5bf841128356e0dae8a3c50e; no voters:
I20260619 08:14:19.332546 3133 leader_election.cc:288] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:19.332571 3138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:19.332675 3138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [term 1 LEADER]: Becoming Leader. State: Replica: 887167dd5bf841128356e0dae8a3c50e, State: Running, Role: LEADER
I20260619 08:14:19.332768 3138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } }
I20260619 08:14:19.332804 3133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:19.333081 3138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "887167dd5bf841128356e0dae8a3c50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } } }
I20260619 08:14:19.333173 3138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:19.333323 3138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 887167dd5bf841128356e0dae8a3c50e. Latest consensus state: current_term: 1 leader_uuid: "887167dd5bf841128356e0dae8a3c50e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887167dd5bf841128356e0dae8a3c50e" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45843 } } }
I20260619 08:14:19.333380 3138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:19.333495 3144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:19.334189 3144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:19.336097 3144 catalog_manager.cc:1347] Generated new cluster ID: 40b4795358bb4bb996860957100d97ff
I20260619 08:14:19.336150 3144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:19.340348 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.329786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.2.119.254:45843)
0619 08:14:19.329953 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.329960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.329983 (+ 23us) server_negotiation.cc:407] Connection header received
0619 08:14:19.330726 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.330731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.330775 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.330885 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.331715 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.332766 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.333985 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.334579 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.334710 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.337130 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.337134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.337134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.337146 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.338832 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.339215 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.339218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.339218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.339328 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.339528 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.339529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.339530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.339777 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.339947 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.340016 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.340150 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260619 08:14:19.349725 3144 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:19.350211 3144 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:19.353163 3144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 887167dd5bf841128356e0dae8a3c50e: Generated new TSK 0
I20260619 08:14:19.353350 3144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:19.388576 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:19.476372 3159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:19.476569 3159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:19.476593 3159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:19.477814 3159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:19.477852 3159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:19.477880 3159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:19.479363 3159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:19.479604 3159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:19.479835 3159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:19.484884 3165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.484882 3164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:19.485139 3159 server_base.cc:1034] running on GCE node
W20260619 08:14:19.484920 3167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:19.485428 3159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:19.485664 3159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:19.486874 3159 hybrid_clock.cc:648] HybridClock initialized: now 1781856859486863 us; error 28 us; skew 500 ppm
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:19.489447 3159 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:19.489954 3159 webserver.cc:460] Webserver started at http://127.2.119.193:41777/ using document root <none> and password file <none>
I20260619 08:14:19.490176 3159 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:19.490249 3159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:19.490381 3159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:19.491420 3159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "d68ea852f521448983ca0e4a9d524440"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "1da7731d9b76a4271c8ec6fb8caeded0"
server_key_iv: "32f4369c7df73710ba03983d51c3c476"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.491777 3159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "d68ea852f521448983ca0e4a9d524440"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "1da7731d9b76a4271c8ec6fb8caeded0"
server_key_iv: "32f4369c7df73710ba03983d51c3c476"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.493083 3159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:19.493885 3174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.494059 3159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.494149 3159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "d68ea852f521448983ca0e4a9d524440"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "1da7731d9b76a4271c8ec6fb8caeded0"
server_key_iv: "32f4369c7df73710ba03983d51c3c476"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.494230 3159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:19.528993 3159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:19.531297 3159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:19.531512 3159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:19.531805 3159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:19.532131 3159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:19.532184 3159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.532228 3159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:19.532254 3159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.538268 3159 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:37019
I20260619 08:14:19.538343 3287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:37019 every 8 connection(s)
I20260619 08:14:19.538695 3159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:19.544051 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3159
I20260619 08:14:19.544131 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:19.544303 2527 external_mini_cluster.cc:1428] Setting key 378d5937b15c8e0d36a4ecd1a684f4fa
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:19.548583 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.539399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:45869 (local address 127.2.119.254:45843)
0619 08:14:19.539468 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.539470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.539703 (+ 233us) server_negotiation.cc:407] Connection header received
0619 08:14:19.540700 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.540701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.540768 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.540819 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.541471 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.541982 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.543033 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.543251 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.543336 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.545634 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.545636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.545636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.545645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.547131 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.547571 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.547572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.547573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.547639 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.547874 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.547875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.547876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.548021 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.548090 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.548334 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.548437 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:19.548681 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.539411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45843 (local address 127.2.119.193:45869)
0619 08:14:19.539582 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.539618 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.540587 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.540892 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.540896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.541154 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:19.541374 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.541378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.542083 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.542084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.542937 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.542940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.543346 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.543347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.543471 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.543928 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:19.543936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:19.545539 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:19.547210 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.547216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.547217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.547480 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.547709 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.547710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.547710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.547812 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.548084 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:19.548088 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:19.548204 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.548287 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.548467 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 08:14:19.549157 3288 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45843
I20260619 08:14:19.549265 3288 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:19.549466 3288 heartbeater.cc:511] Master 127.2.119.254:45843 requested a full tablet report, sending...
I20260619 08:14:19.550072 3097 ts_manager.cc:194] Registered new tserver with Master: d68ea852f521448983ca0e4a9d524440 (127.2.119.193:37019)
I20260619 08:14:19.551024 3097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:45869
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:19.570412 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:19.650009 3293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:19.650207 3293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:19.650230 3293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:19.651369 3293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:19.651412 3293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:19.651437 3293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:19.652921 3293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:19.653177 3293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:19.653420 3293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:19.658609 3301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.658619 3298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.658620 3299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:19.658855 3293 server_base.cc:1034] running on GCE node
I20260619 08:14:19.659150 3293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:19.659399 3293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:19.660579 3293 hybrid_clock.cc:648] HybridClock initialized: now 1781856859660569 us; error 48 us; skew 500 ppm
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:19.663323 3293 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:19.663853 3293 webserver.cc:460] Webserver started at http://127.2.119.194:36033/ using document root <none> and password file <none>
I20260619 08:14:19.664090 3293 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:19.664155 3293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:19.664311 3293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:19.665439 3293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "4f16397b7f62488da5275e891b7aeeae"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "e9b89d9871ffd25b1b3b13b3b99cb912"
server_key_iv: "c437d5553ec94cc64123551585939c68"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.665807 3293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "4f16397b7f62488da5275e891b7aeeae"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "e9b89d9871ffd25b1b3b13b3b99cb912"
server_key_iv: "c437d5553ec94cc64123551585939c68"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.667181 3293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:19.667958 3308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.668239 3293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:19.668336 3293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "4f16397b7f62488da5275e891b7aeeae"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "e9b89d9871ffd25b1b3b13b3b99cb912"
server_key_iv: "c437d5553ec94cc64123551585939c68"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.668416 3293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:19.684309 3293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:19.686290 3293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:19.686445 3293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:19.686667 3293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:19.686986 3293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:19.687036 3293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.687077 3293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:19.687104 3293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.693017 3293 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:34253
I20260619 08:14:19.693070 3421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:34253 every 8 connection(s)
I20260619 08:14:19.693462 3293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:19.694721 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3293
I20260619 08:14:19.694818 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:19.695026 2527 external_mini_cluster.cc:1428] Setting key c392b7b25bd5f8713111399993b69338
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:19.704347 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.694172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45843 (local address 127.2.119.194:38245)
0619 08:14:19.694323 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.694357 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.695647 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.695910 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.695915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.696315 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:19.696613 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.696620 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.697489 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.697490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.698523 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.698526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.699059 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.699059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.699171 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.699673 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:19.699682 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:19.701057 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:19.702959 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.702965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.702966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.703180 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.703402 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.703403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.703403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.703496 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.703783 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:19.703787 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:19.703907 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.703991 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.704109 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260619 08:14:19.704360 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.694190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:38245 (local address 127.2.119.254:45843)
0619 08:14:19.694255 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.694257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.694453 (+ 196us) server_negotiation.cc:407] Connection header received
0619 08:14:19.695746 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.695747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.695759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.695833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.696714 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.697380 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.698632 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.698976 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.699054 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.701149 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.701151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.701151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.701169 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.702876 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.703257 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.703259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.703259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.703325 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.703561 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.703562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.703563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.703714 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.703799 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.704044 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.704143 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:14:19.704941 3422 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45843
I20260619 08:14:19.705058 3422 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:19.705229 3422 heartbeater.cc:511] Master 127.2.119.254:45843 requested a full tablet report, sending...
I20260619 08:14:19.705631 3097 ts_manager.cc:194] Registered new tserver with Master: 4f16397b7f62488da5275e891b7aeeae (127.2.119.194:34253)
I20260619 08:14:19.706075 3097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:38245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:19.725175 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:19.806402 3427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:19.806638 3427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:19.806674 3427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:19.807819 3427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:19.807884 3427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:19.807951 3427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:19.809540 3427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:19.809809 3427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:19.810026 3427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:19.815129 3435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.815256 3432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:19.815128 3433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:19.815297 3427 server_base.cc:1034] running on GCE node
I20260619 08:14:19.815563 3427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:19.815860 3427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:19.817054 3427 hybrid_clock.cc:648] HybridClock initialized: now 1781856859817051 us; error 42 us; skew 500 ppm
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:19.819819 3427 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:19.820322 3427 webserver.cc:460] Webserver started at http://127.2.119.195:42301/ using document root <none> and password file <none>
I20260619 08:14:19.820573 3427 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:19.820638 3427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:19.820807 3427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:19.821952 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "78d6415b0e344f84b88418b379e3c066"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "df8a4895cf87f829e62c8bda1c5fe3ca"
server_key_iv: "c0cd2a9a445005be78979636ca02b33e"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.822330 3427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "78d6415b0e344f84b88418b379e3c066"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "df8a4895cf87f829e62c8bda1c5fe3ca"
server_key_iv: "c0cd2a9a445005be78979636ca02b33e"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.823640 3427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:19.824411 3442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.824581 3427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:19.824684 3427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "78d6415b0e344f84b88418b379e3c066"
format_stamp: "Formatted at 2026-06-19 08:14:19 on dist-test-slave-d0b5"
server_key: "df8a4895cf87f829e62c8bda1c5fe3ca"
server_key_iv: "c0cd2a9a445005be78979636ca02b33e"
server_key_version: "encryptionkey@0"
I20260619 08:14:19.824813 3427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:19.852372 3427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:19.854704 3427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:19.854871 3427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:19.855103 3427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:19.855425 3427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:19.855477 3427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.855517 3427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:19.855545 3427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:19.861594 3427 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44125
I20260619 08:14:19.861645 3555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44125 every 8 connection(s)
I20260619 08:14:19.861984 3427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:19.870386 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3427
I20260619 08:14:19.870498 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:19.870684 2527 external_mini_cluster.cc:1428] Setting key f5a062bfe5add203cc06a1f03675c9e0
I20260619 08:14:19.871800 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.862731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:35631 (local address 127.2.119.254:45843)
0619 08:14:19.862789 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.862791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.863020 (+ 229us) server_negotiation.cc:407] Connection header received
0619 08:14:19.863928 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.863929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.863940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.863999 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.864649 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.865153 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.866169 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.866364 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.866415 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.868604 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.868605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.868606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.868613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.870144 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.870686 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.870687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.870687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.870806 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.871132 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.871134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.871134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.871262 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.871339 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.871628 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.871709 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:14:19.872037 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.862706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45843 (local address 127.2.119.195:35631)
0619 08:14:19.862862 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.862908 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.863821 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.864089 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.864093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.864343 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:19.864547 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.864551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.865234 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.865235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.866089 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.866092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.866447 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.866449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.866561 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.867113 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:19.867121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:19.868505 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:19.870239 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.870247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.870250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.870592 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.870884 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:19.870885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:19.870885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:19.871044 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:19.871331 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:19.871337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:19.871489 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.871595 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.871759 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260619 08:14:19.872447 3556 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45843
I20260619 08:14:19.872535 3556 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:19.872675 3556 heartbeater.cc:511] Master 127.2.119.254:45843 requested a full tablet report, sending...
I20260619 08:14:19.873086 3097 ts_manager.cc:194] Registered new tserver with Master: 78d6415b0e344f84b88418b379e3c066 (127.2.119.195:44125)
I20260619 08:14:19.873569 3097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:35631
I20260619 08:14:19.882603 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:19.890148 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.884155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.2.119.254:45843)
0619 08:14:19.884234 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.884236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.884296 (+ 60us) server_negotiation.cc:407] Connection header received
0619 08:14:19.884339 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.884340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.884349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.884405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.884899 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.885457 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.886176 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.886364 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.886421 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.887498 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.887500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.887500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.887506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.888751 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.889070 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.889071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.889072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.889124 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.889417 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.889418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.889419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.889566 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.889656 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.889843 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.890032 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:14:19.892329 3097 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57524:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:19.894070 3097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:19.900655 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:34253 (local address 127.0.0.1:55546)
0619 08:14:19.896701 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.896741 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.896819 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.897153 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.897157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.897160 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:19.897243 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.897245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.898308 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.898309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.899442 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.899444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.900350 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.900429 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.900443 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.900520 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.900573 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":56,"threads_started":1}
I20260619 08:14:19.900681 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.2.119.194:34253)
0619 08:14:19.896692 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.896698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.896797 (+ 99us) server_negotiation.cc:407] Connection header received
0619 08:14:19.896908 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.896909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.896923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.897032 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:19.897487 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.898211 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.899539 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.900271 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900342 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.900396 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.900550 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.900592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260619 08:14:19.901069 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:37019 (local address 127.0.0.1:57286)
0619 08:14:19.897055 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.897065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.897114 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.897288 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.897289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.897290 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:19.897377 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.897378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.899029 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.899030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.899961 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.899963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.900828 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.900892 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.900896 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.900968 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.901006 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":5,"thread_start_us":65,"threads_started":1}
I20260619 08:14:19.901095 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.2.119.193:37019)
0619 08:14:19.897063 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.897067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.897114 (+ 47us) server_negotiation.cc:407] Connection header received
0619 08:14:19.897178 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.897180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.897195 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.897283 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:19.897656 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.898931 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900046 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.900657 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900734 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.900792 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.900983 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.901024 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260619 08:14:19.901405 3356 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12a0abdfda452c9cec650dd0ea779d (DEFAULT_TABLE table=test-table [id=7bb1f6e0142443d2b2b1d19981f2fb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:19.901615 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44125 (local address 127.0.0.1:37634)
0619 08:14:19.896941 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.896950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.896994 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.897320 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.897321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.897323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:19.897377 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.897378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.898835 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.898836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.899915 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.899917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.901384 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.901385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.901454 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.901457 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.901500 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.901528 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":40,"thread_start_us":64,"threads_started":1}
I20260619 08:14:19.901782 3356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12a0abdfda452c9cec650dd0ea779d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:19.901827 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.896573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37634 (local address 127.2.119.195:44125)
0619 08:14:19.896740 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.896744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.897049 (+ 305us) server_negotiation.cc:407] Connection header received
0619 08:14:19.897094 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.897097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.897113 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.897183 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:19.897471 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.898740 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.900004 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.901051 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.901364 (+ 313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.901426 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.901512 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.901553 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260619 08:14:19.902347 3490 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12a0abdfda452c9cec650dd0ea779d (DEFAULT_TABLE table=test-table [id=7bb1f6e0142443d2b2b1d19981f2fb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:19.902616 3490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12a0abdfda452c9cec650dd0ea779d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:19.904372 3573 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Bootstrap starting.
I20260619 08:14:19.905099 3573 tablet_bootstrap.cc:654] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:19.905051 3222 tablet_service.cc:1467] Processing CreateTablet for tablet 3e12a0abdfda452c9cec650dd0ea779d (DEFAULT_TABLE table=test-table [id=7bb1f6e0142443d2b2b1d19981f2fb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:19.905202 3574 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: Bootstrap starting.
I20260619 08:14:19.905381 3222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e12a0abdfda452c9cec650dd0ea779d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:19.905915 3573 log.cc:824] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:19.906102 3574 tablet_bootstrap.cc:654] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:19.906631 3574 log.cc:824] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:19.906741 3573 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: No bootstrap required, opened a new log
I20260619 08:14:19.906817 3573 ts_tablet_manager.cc:1397] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:14:19.907405 3574 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: No bootstrap required, opened a new log
I20260619 08:14:19.907460 3574 ts_tablet_manager.cc:1397] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:19.908022 3577 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: Bootstrap starting.
I20260619 08:14:19.908689 3577 tablet_bootstrap.cc:654] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:19.908849 3573 raft_consensus.cc:348] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.909036 3573 raft_consensus.cc:374] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:19.909082 3573 raft_consensus.cc:729] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f16397b7f62488da5275e891b7aeeae, State: Initialized, Role: FOLLOWER
I20260619 08:14:19.909227 3577 log.cc:824] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:19.909229 3573 consensus_queue.cc:260] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.909543 3422 heartbeater.cc:503] Master 127.2.119.254:45843 was elected leader, sending a full tablet report...
I20260619 08:14:19.909531 3573 ts_tablet_manager.cc:1428] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 08:14:19.909931 3577 tablet_bootstrap.cc:492] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: No bootstrap required, opened a new log
I20260619 08:14:19.909991 3577 ts_tablet_manager.cc:1397] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:14:19.909833 3574 raft_consensus.cc:348] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.910030 3574 raft_consensus.cc:374] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:19.910058 3574 raft_consensus.cc:729] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d6415b0e344f84b88418b379e3c066, State: Initialized, Role: FOLLOWER
I20260619 08:14:19.910164 3574 consensus_queue.cc:260] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.910420 3574 ts_tablet_manager.cc:1428] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:19.910418 3556 heartbeater.cc:503] Master 127.2.119.254:45843 was elected leader, sending a full tablet report...
I20260619 08:14:19.911800 3577 raft_consensus.cc:348] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.911909 3577 raft_consensus.cc:374] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:19.911929 3577 raft_consensus.cc:729] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d68ea852f521448983ca0e4a9d524440, State: Initialized, Role: FOLLOWER
I20260619 08:14:19.912053 3577 consensus_queue.cc:260] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.912304 3577 ts_tablet_manager.cc:1428] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:14:19.912461 3288 heartbeater.cc:503] Master 127.2.119.254:45843 was elected leader, sending a full tablet report...
W20260619 08:14:19.944062 3423 tablet.cc:2369] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:19.957578 3578 raft_consensus.cc:482] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:19.957715 3578 raft_consensus.cc:504] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.958151 3578 leader_election.cc:288] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78d6415b0e344f84b88418b379e3c066 (127.2.119.195:44125), d68ea852f521448983ca0e4a9d524440 (127.2.119.193:37019)
I20260619 08:14:19.961153 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.958508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:53925 (local address 127.2.119.193:37019)
0619 08:14:19.958592 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.958594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.958675 (+ 81us) server_negotiation.cc:407] Connection header received
0619 08:14:19.958732 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.958733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.958756 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.958808 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:19.958970 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.959439 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960404 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.960803 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960889 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.960897 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.961027 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.961076 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:14:19.961177 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.958428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:45975 (local address 127.2.119.195:44125)
0619 08:14:19.958478 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.958479 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.958489 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:19.958518 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.958519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.958529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.958596 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:19.958823 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.959364 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960426 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.960804 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960882 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.960890 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.961080 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.961118 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:19.961169 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.958438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:37019 (local address 127.2.119.194:53925)
0619 08:14:19.958599 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.958610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.958668 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.958834 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.958835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.958837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:19.958901 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.958902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.959526 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.959527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.960313 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.960315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.960885 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.960958 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.960962 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.961005 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.961034 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260619 08:14:19.961175 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.958330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44125 (local address 127.2.119.194:45975)
0619 08:14:19.958377 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:19.958391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:19.958435 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:19.958641 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:19.958641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:19.958643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:19.958735 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.958736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.959450 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.959451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.960316 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:19.960318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.960898 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.960899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:19.961002 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.961006 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:19.961059 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:14:19.961095 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":66}
I20260619 08:14:19.961565 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12a0abdfda452c9cec650dd0ea779d" candidate_uuid: "4f16397b7f62488da5275e891b7aeeae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d68ea852f521448983ca0e4a9d524440" is_pre_election: true
I20260619 08:14:19.961750 3242 raft_consensus.cc:2454] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f16397b7f62488da5275e891b7aeeae in term 0.
I20260619 08:14:19.961784 3510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12a0abdfda452c9cec650dd0ea779d" candidate_uuid: "4f16397b7f62488da5275e891b7aeeae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d6415b0e344f84b88418b379e3c066" is_pre_election: true
I20260619 08:14:19.961953 3510 raft_consensus.cc:2454] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f16397b7f62488da5275e891b7aeeae in term 0.
I20260619 08:14:19.961983 3309 leader_election.cc:302] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f16397b7f62488da5275e891b7aeeae, d68ea852f521448983ca0e4a9d524440; no voters:
I20260619 08:14:19.962108 3578 raft_consensus.cc:2781] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:19.962191 3578 raft_consensus.cc:482] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:19.962225 3578 raft_consensus.cc:3037] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:19.962913 3578 raft_consensus.cc:504] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.963039 3578 leader_election.cc:288] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [CANDIDATE]: Term 1 election: Requested vote from peers 78d6415b0e344f84b88418b379e3c066 (127.2.119.195:44125), d68ea852f521448983ca0e4a9d524440 (127.2.119.193:37019)
I20260619 08:14:19.963208 3510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12a0abdfda452c9cec650dd0ea779d" candidate_uuid: "4f16397b7f62488da5275e891b7aeeae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d6415b0e344f84b88418b379e3c066"
I20260619 08:14:19.963232 3242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e12a0abdfda452c9cec650dd0ea779d" candidate_uuid: "4f16397b7f62488da5275e891b7aeeae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d68ea852f521448983ca0e4a9d524440"
I20260619 08:14:19.963299 3510 raft_consensus.cc:3037] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:19.963306 3242 raft_consensus.cc:3037] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:19.964016 3510 raft_consensus.cc:2454] T 3e12a0abdfda452c9cec650dd0ea779d P 78d6415b0e344f84b88418b379e3c066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f16397b7f62488da5275e891b7aeeae in term 1.
I20260619 08:14:19.964020 3242 raft_consensus.cc:2454] T 3e12a0abdfda452c9cec650dd0ea779d P d68ea852f521448983ca0e4a9d524440 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f16397b7f62488da5275e891b7aeeae in term 1.
I20260619 08:14:19.964205 3309 leader_election.cc:302] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f16397b7f62488da5275e891b7aeeae, d68ea852f521448983ca0e4a9d524440; no voters:
I20260619 08:14:19.964319 3578 raft_consensus.cc:2781] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:19.964462 3578 raft_consensus.cc:686] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [term 1 LEADER]: Becoming Leader. State: Replica: 4f16397b7f62488da5275e891b7aeeae, State: Running, Role: LEADER
I20260619 08:14:19.964555 3578 consensus_queue.cc:237] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } }
I20260619 08:14:19.965438 3096 catalog_manager.cc:5495] T 3e12a0abdfda452c9cec650dd0ea779d P 4f16397b7f62488da5275e891b7aeeae reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f16397b7f62488da5275e891b7aeeae (127.2.119.194). New cstate: current_term: 1 leader_uuid: "4f16397b7f62488da5275e891b7aeeae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f16397b7f62488da5275e891b7aeeae" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 34253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78d6415b0e344f84b88418b379e3c066" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d68ea852f521448983ca0e4a9d524440" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37019 } health_report { overall_health: UNKNOWN } } }
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:19.977496 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.969955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.2.119.193:37019)
0619 08:14:19.970016 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.970018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.970128 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:14:19.970208 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.970209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.970220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.970304 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.970979 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.971449 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.972256 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.972456 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.972525 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.974918 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.974930 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.974931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.974938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.976339 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.976675 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.976679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.976679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.976737 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:19.976934 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:19.976935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:19.976936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:19.977078 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:19.977234 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:19.977289 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:14:19.977392 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:14:19 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:20.000744 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:19.993274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.2.119.193:37019)
0619 08:14:19.993564 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:19.993567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:19.993666 (+ 99us) server_negotiation.cc:407] Connection header received
0619 08:14:19.993759 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:19.993760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:19.993775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:19.993845 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:19.994415 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.994896 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.995604 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:19.995799 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:19.995879 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:19.998378 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:19.998379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:19.998380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:19.998387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:19.999726 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:20.000084 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:20.000085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:20.000085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:20.000138 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:20.000295 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:20.000296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:20.000296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:20.000390 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:20.000465 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:20.000607 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 08:14:20.000673 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
Jun 19 08:14:20 dist-test-slave-d0b5 krb5kdc[3052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:14:20 dist-test-slave-d0b5 krb5kdc[3052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:20.023552 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:20.015993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.2.119.193:37019)
0619 08:14:20.016061 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:20.016063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:20.016103 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:14:20.016170 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:20.016170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:20.016180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:20.016230 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:20.016803 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:20.017268 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:20.018141 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:20.018335 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:20.018401 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:20.020790 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:20.020791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:20.020791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:20.020798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:20.022172 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:20.022564 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:20.022565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:20.022566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:20.022636 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:20.022927 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:20.022928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:20.022928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:20.023028 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:20.023104 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:20.023306 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 08:14:20.023428 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 08:14:20.024163 3202 tablet_service.cc:2908] Rejecting scan request for tablet 3e12a0abdfda452c9cec650dd0ea779d: Uninitialized: clean time has not yet been initialized
I20260619 08:14:20.025038 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3159
I20260619 08:14:20.030213 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3293
I20260619 08:14:20.034828 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3427
I20260619 08:14:20.039104 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3065
2026-06-19T08:14:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2021 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:20 dist-test-slave-d0b5 krb5kdc[3597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:20 dist-test-slave-d0b5 krb5kdc[3597](info): set up 2 sockets
Jun 19 08:14:20 dist-test-slave-d0b5 krb5kdc[3597](info): commencing operation
krb5kdc: starting...
W20260619 08:14:21.138010 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:21.256551 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45223
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:21.336956 3610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:21.337182 3610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:21.337220 3610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:21.338400 3610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:21.338467 3610 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:14:21.338492 3610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:21.338524 3610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:21.338565 3610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:21.338595 3610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:21.340060 3610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45223
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:21.340323 3610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:21.340541 3610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:21.345672 3615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:21.345659 3616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.345742 3610 server_base.cc:1034] running on GCE node
W20260619 08:14:21.345798 3618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.346213 3610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:21.346467 3610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:21.347607 3610 hybrid_clock.cc:648] HybridClock initialized: now 1781856861347590 us; error 34 us; skew 500 ppm
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:21.353883 3610 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:21.354384 3610 webserver.cc:460] Webserver started at http://127.2.119.254:42861/ using document root <none> and password file <none>
I20260619 08:14:21.354610 3610 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:21.354691 3610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:21.354882 3610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:21.355968 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "417d64089e4346e9be9f3121a3051627"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "e7c9da568b784fb13966f0fea5f9c7e7"
server_key_iv: "d4a4901502c06df125ab060c7dae7bb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.356343 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "417d64089e4346e9be9f3121a3051627"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "e7c9da568b784fb13966f0fea5f9c7e7"
server_key_iv: "d4a4901502c06df125ab060c7dae7bb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.357769 3610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:21.358564 3625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.358752 3610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:21.358850 3610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "417d64089e4346e9be9f3121a3051627"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "e7c9da568b784fb13966f0fea5f9c7e7"
server_key_iv: "d4a4901502c06df125ab060c7dae7bb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.358990 3610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:21.369624 3610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:21.371829 3610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:21.372030 3610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:21.377000 3610 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45223
I20260619 08:14:21.377053 3687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45223 every 8 connection(s)
I20260619 08:14:21.377475 3610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:21.378129 3688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:21.380391 3688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Bootstrap starting.
I20260619 08:14:21.380985 3688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:21.381285 3688 log.cc:824] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:21.381503 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3610
I20260619 08:14:21.381628 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:21.381812 2527 external_mini_cluster.cc:1428] Setting key cde3f07ca152659b134cdad48fd3edcd
I20260619 08:14:21.382156 3688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: No bootstrap required, opened a new log
I20260619 08:14:21.384809 3688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } }
I20260619 08:14:21.384980 3688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:21.385020 3688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417d64089e4346e9be9f3121a3051627, State: Initialized, Role: FOLLOWER
I20260619 08:14:21.385133 3688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } }
I20260619 08:14:21.385205 3688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:21.385239 3688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:21.385282 3688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:21.385964 3688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } }
I20260619 08:14:21.386077 3688 leader_election.cc:302] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 417d64089e4346e9be9f3121a3051627; no voters:
I20260619 08:14:21.386261 3688 leader_election.cc:288] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:21.386315 3693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:21.386511 3693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [term 1 LEADER]: Becoming Leader. State: Replica: 417d64089e4346e9be9f3121a3051627, State: Running, Role: LEADER
I20260619 08:14:21.386533 3688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:21.386669 3693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } }
I20260619 08:14:21.387096 3694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "417d64089e4346e9be9f3121a3051627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } } }
I20260619 08:14:21.387185 3694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:21.387351 3695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 417d64089e4346e9be9f3121a3051627. Latest consensus state: current_term: 1 leader_uuid: "417d64089e4346e9be9f3121a3051627" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417d64089e4346e9be9f3121a3051627" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45223 } } }
I20260619 08:14:21.387396 3697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:21.387411 3695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:21.387892 3697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:21.389484 3697 catalog_manager.cc:1347] Generated new cluster ID: 57593bf246a14d0e81d92b595f6918a5
I20260619 08:14:21.389539 3697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:21.394630 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.382303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55296 (local address 127.2.119.254:45223)
0619 08:14:21.382497 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.382507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.382540 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:14:21.383456 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.383465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.383527 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.383689 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.384537 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.385959 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.387081 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.387675 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.387781 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.390948 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.390952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.390952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.390963 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.392855 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.393224 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.393227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.393228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.393361 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.393534 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.393534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.393534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.393800 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.393931 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.394308 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.394443 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260619 08:14:21.406472 3697 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:21.407217 3697 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:21.412406 3697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Generated new TSK 0
I20260619 08:14:21.412580 3697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:21.440156 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45223
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:21.521461 3714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:21.521668 3714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:21.521693 3714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:21.522832 3714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:21.522869 3714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:21.522897 3714 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:21.522914 3714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:21.524536 3714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:21.524736 3714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:21.524914 3714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:21.530489 3720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.530601 3714 server_base.cc:1034] running on GCE node
W20260619 08:14:21.530514 3722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:21.530687 3719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.531181 3714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:21.531468 3714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:21.532646 3714 hybrid_clock.cc:648] HybridClock initialized: now 1781856861532621 us; error 44 us; skew 500 ppm
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:21.535399 3714 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:21.535887 3714 webserver.cc:460] Webserver started at http://127.2.119.193:39621/ using document root <none> and password file <none>
I20260619 08:14:21.536145 3714 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:21.536218 3714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:21.536353 3714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:21.537498 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "7b78943e42b444118a35568a8dabe1cb"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "2652e3ed2897477c6fc30170d694961c"
server_key_iv: "4f59cacd525f5e5fd71073c3f130f47a"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.537904 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "7b78943e42b444118a35568a8dabe1cb"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "2652e3ed2897477c6fc30170d694961c"
server_key_iv: "4f59cacd525f5e5fd71073c3f130f47a"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.539264 3714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:21.540051 3729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.540237 3714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:21.540333 3714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "7b78943e42b444118a35568a8dabe1cb"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "2652e3ed2897477c6fc30170d694961c"
server_key_iv: "4f59cacd525f5e5fd71073c3f130f47a"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.540416 3714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:21.551115 3714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:21.553515 3714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:21.553678 3714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:21.554178 3714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:21.554239 3714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.554284 3714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:21.554315 3714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.561048 3714 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:37269
I20260619 08:14:21.561172 3844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:37269 every 8 connection(s)
I20260619 08:14:21.561497 3714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:21.564833 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3714
I20260619 08:14:21.564914 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:21.565092 2527 external_mini_cluster.cc:1428] Setting key 0c78c9c702bd6d5645e92b5afcbebc36
I20260619 08:14:21.565261 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.554746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58085 (local address 127.2.119.254:45223)
0619 08:14:21.554833 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.554836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.555008 (+ 172us) server_negotiation.cc:407] Connection header received
0619 08:14:21.555570 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.555572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.555588 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.555667 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.556636 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.557516 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.559184 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.559569 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.559654 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.562394 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.562395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.562395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.562402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.563838 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.564241 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.564241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.564242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.564292 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.564527 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.564527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.564527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.564634 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.564684 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.565050 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.565149 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:14:21.565459 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.554741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.193:58085)
0619 08:14:21.554898 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.554937 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.555424 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.555830 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.555836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.556241 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.556524 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.556531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.557611 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.557612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.559065 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.559068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.559677 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.559679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.559898 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.560521 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.560533 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.562291 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.563921 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.563929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.563931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.564168 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.564368 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.564368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.564369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.564471 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.564706 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.564711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.564832 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.564981 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.565155 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260619 08:14:21.567137 3845 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45223
I20260619 08:14:21.567277 3845 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:21.567514 3845 heartbeater.cc:511] Master 127.2.119.254:45223 requested a full tablet report, sending...
I20260619 08:14:21.568078 3641 ts_manager.cc:194] Registered new tserver with Master: 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269)
I20260619 08:14:21.568747 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:58085
I20260619 08:14:21.573522 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.567060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.193:36041)
0619 08:14:21.567210 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.567221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.567311 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.567467 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.567467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.567660 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.567711 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.567734 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.568588 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.568589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.569293 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.569296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.569865 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.569866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.569915 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.570354 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.570359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.570636 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.572518 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.572519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.572519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.572674 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.572896 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.572896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.572896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.572946 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.573239 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.573240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.573273 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.573385 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.573459 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260619 08:14:21.573630 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.567113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:36041 (local address 127.2.119.254:45223)
0619 08:14:21.567154 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.567157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.567300 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:14:21.567387 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.567388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.567399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.567461 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.567880 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.568491 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.569405 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.569775 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.569852 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.570725 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.570726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.570727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.570735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.572433 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.572751 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.572753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.572753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.572820 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.573013 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.573014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.573015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.573160 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.573238 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.573418 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.573504 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:21.594568 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45223
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:21.676608 3854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:21.676939 3854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:21.676977 3854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:21.678205 3854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:21.678280 3854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:21.678318 3854 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:21.678355 3854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:21.679858 3854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:21.680122 3854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:21.680358 3854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:21.685506 3862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:21.685597 3859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:21.685581 3860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.685626 3854 server_base.cc:1034] running on GCE node
I20260619 08:14:21.686010 3854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:21.686213 3854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:21.687335 3854 hybrid_clock.cc:648] HybridClock initialized: now 1781856861687322 us; error 32 us; skew 500 ppm
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:21.690106 3854 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:21.690580 3854 webserver.cc:460] Webserver started at http://127.2.119.194:42713/ using document root <none> and password file <none>
I20260619 08:14:21.690838 3854 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:21.690907 3854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:21.691053 3854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:21.692101 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "40175f2a7af9409a814a97227cdef9f7"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "7099555cbd72e65f022ca0c4b5f6cf9e"
server_key_iv: "ef58687b1634516715f6b7037043595e"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.692467 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "40175f2a7af9409a814a97227cdef9f7"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "7099555cbd72e65f022ca0c4b5f6cf9e"
server_key_iv: "ef58687b1634516715f6b7037043595e"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.693946 3854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:21.694799 3869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.695125 3854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:21.695220 3854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "40175f2a7af9409a814a97227cdef9f7"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "7099555cbd72e65f022ca0c4b5f6cf9e"
server_key_iv: "ef58687b1634516715f6b7037043595e"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.695334 3854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:21.710276 3854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:21.712620 3854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:21.712788 3854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:21.713375 3854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:21.713435 3854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.713476 3854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:21.713505 3854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.720754 3854 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:43145
I20260619 08:14:21.720836 3984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:43145 every 8 connection(s)
I20260619 08:14:21.721138 3854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:21.724593 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.713875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:43591 (local address 127.2.119.254:45223)
0619 08:14:21.713945 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.713947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.714159 (+ 212us) server_negotiation.cc:407] Connection header received
0619 08:14:21.714747 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.714749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.714763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.714821 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.715819 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.716766 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.718523 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.718815 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.718876 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.721777 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.721778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.721779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.721786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.723150 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.723587 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.723588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.723588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.723646 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.723896 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.723897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.723897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.724085 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.724165 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.724440 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.724515 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:14:21.724777 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.713868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.194:43591)
0619 08:14:21.714031 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.714068 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.714606 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.714896 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.714900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.715300 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.715604 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.715610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.716878 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.716879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.718397 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.718402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.718909 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.718910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.719083 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.719797 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.719816 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.721673 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.723245 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.723251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.723253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.723506 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.723714 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.723715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.723715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.723830 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.724159 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.724164 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.724307 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.724401 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.724532 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260619 08:14:21.725548 3985 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45223
I20260619 08:14:21.725639 3985 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:21.725904 3985 heartbeater.cc:511] Master 127.2.119.254:45223 requested a full tablet report, sending...
I20260619 08:14:21.726361 3641 ts_manager.cc:194] Registered new tserver with Master: 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145)
I20260619 08:14:21.726939 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:43591
I20260619 08:14:21.729539 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3854
I20260619 08:14:21.729651 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:21.729851 2527 external_mini_cluster.cc:1428] Setting key 5ab37f769758cc7528068aee9fdce5b4
I20260619 08:14:21.733640 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.726748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:56027 (local address 127.2.119.254:45223)
0619 08:14:21.726810 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.726812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.727244 (+ 432us) server_negotiation.cc:407] Connection header received
0619 08:14:21.727310 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.727311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.727320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.727403 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.727735 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.728485 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.729418 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.729670 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.729731 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.730734 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.730735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.730736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.730744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.732383 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.732808 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.732809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.732810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.732870 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.733091 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.733092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.733092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.733246 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.733363 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.733498 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.733579 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:21.733747 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.726642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.194:56027)
0619 08:14:21.727145 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.727155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.727249 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.727395 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.727395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.727581 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.727653 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.727655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.728574 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.728575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.729277 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.729280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.729798 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.729799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.729856 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.730354 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.730359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.730649 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.732467 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.732468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.732468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.732708 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.732944 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.732945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.732945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.733020 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.733335 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.733336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.733388 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.733466 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.733593 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":2,"thread_start_us":145,"threads_started":1}
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:21.756846 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45223
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:21.838527 3994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:21.838722 3994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:21.838745 3994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:21.839823 3994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:21.839860 3994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:21.839880 3994 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:21.839897 3994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:21.841336 3994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:21.841511 3994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:21.841675 3994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:21.846942 4000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:21.846939 3999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.847224 3994 server_base.cc:1034] running on GCE node
W20260619 08:14:21.846964 4002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:21.847638 3994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:21.847908 3994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:21.849094 3994 hybrid_clock.cc:648] HybridClock initialized: now 1781856861849058 us; error 51 us; skew 500 ppm
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:21.851611 3994 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:21.852038 3994 webserver.cc:460] Webserver started at http://127.2.119.195:38309/ using document root <none> and password file <none>
I20260619 08:14:21.852277 3994 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:21.852321 3994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:21.852468 3994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:21.853616 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "8e3d66669b25466ab5a4fb1ac18defa5"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "3987fb5e5c12b674743fa19f9414b5ec"
server_key_iv: "4e9b2300e9ba5e95ced8171bc0de0c73"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.853971 3994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "8e3d66669b25466ab5a4fb1ac18defa5"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "3987fb5e5c12b674743fa19f9414b5ec"
server_key_iv: "4e9b2300e9ba5e95ced8171bc0de0c73"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.855291 3994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:21.856062 4009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.856269 3994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:21.856357 3994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "8e3d66669b25466ab5a4fb1ac18defa5"
format_stamp: "Formatted at 2026-06-19 08:14:21 on dist-test-slave-d0b5"
server_key: "3987fb5e5c12b674743fa19f9414b5ec"
server_key_iv: "4e9b2300e9ba5e95ced8171bc0de0c73"
server_key_version: "encryptionkey@0"
I20260619 08:14:21.856434 3994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:21.863587 3994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:21.865549 3994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:21.865702 3994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:21.866287 3994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:21.866325 3994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.866345 3994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:21.866358 3994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:21.872699 3994 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:41147
I20260619 08:14:21.872864 4124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:41147 every 8 connection(s)
I20260619 08:14:21.873236 3994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:21.876973 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.866995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:38017 (local address 127.2.119.254:45223)
0619 08:14:21.867065 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.867068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.867257 (+ 189us) server_negotiation.cc:407] Connection header received
0619 08:14:21.867815 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.867816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.867830 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.867916 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.868739 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.869617 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.871178 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.871535 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.871618 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.874288 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.874289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.874290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.874297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.875731 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.876110 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.876111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.876111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.876161 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.876361 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.876361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.876361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.876479 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.876530 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.876847 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.876910 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:14:21.877159 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.866964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.195:38017)
0619 08:14:21.867132 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.867171 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.867689 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.867989 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.867992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.868360 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.868634 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.868639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.869707 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.869708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.871077 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.871080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.871624 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.871625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.871805 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.872364 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.872372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.874193 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.875819 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.875825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.875826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.876042 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.876216 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.876216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.876216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.876309 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.876555 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.876561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.876676 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.876833 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.876956 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260619 08:14:21.877671 4125 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45223
I20260619 08:14:21.877754 4125 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:21.877948 4125 heartbeater.cc:511] Master 127.2.119.254:45223 requested a full tablet report, sending...
I20260619 08:14:21.878297 3641 ts_manager.cc:194] Registered new tserver with Master: 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195:41147)
I20260619 08:14:21.878772 3641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:38017
I20260619 08:14:21.881867 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 3994
I20260619 08:14:21.881966 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:21.882134 2527 external_mini_cluster.cc:1428] Setting key 13add17476389c5e5e158bb5be3e9fc6
I20260619 08:14:21.882980 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:21.884268 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.878763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:33179 (local address 127.2.119.254:45223)
0619 08:14:21.878807 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.878809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.878875 (+ 66us) server_negotiation.cc:407] Connection header received
0619 08:14:21.878965 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.878966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.878976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.879022 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.879307 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.879833 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.880491 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.880662 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.880709 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.881558 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.881560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.881560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.881566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.883143 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.883489 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.883491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.883491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.883547 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.883759 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.883760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.883760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.883876 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.883947 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.884110 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.884178 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:21.884281 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.878673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.2.119.195:33179)
0619 08:14:21.878807 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.878818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.878901 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.879033 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.879033 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.879190 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:21.879238 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.879239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.879922 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.879923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.880422 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.880423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.880722 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.880723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.880771 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.881121 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:21.881125 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:21.881481 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:21.883233 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.883233 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.883234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.883418 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.883637 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:21.883638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:21.883639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:21.883697 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:21.883949 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:21.883951 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:21.883996 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.884072 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.884177 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:14:21 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:21.907130 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.899918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.2.119.254:45223)
0619 08:14:21.899970 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.899972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.899978 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:14:21.900000 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.900001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.900009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.900050 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:21.900633 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.901119 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.902006 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.902203 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.902261 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.904609 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:21.904610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:21.904610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:21.904618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:21.906050 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.906400 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.906402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.906402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.906461 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:21.906638 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:21.906639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:21.906639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:21.906751 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:21.906838 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.906990 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.907052 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:14:21.908675 3641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:21.910411 3641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:21.916950 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.912516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:43145 (local address 127.0.0.1:55616)
0619 08:14:21.913172 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.913217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.913296 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.913805 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.913808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.913811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:21.913887 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.913889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.914895 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.914897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.915893 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.915896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.916612 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.916613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.916695 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.916714 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.916808 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.916866 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":59,"threads_started":1}
I20260619 08:14:21.916966 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.913444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.2.119.194:43145)
0619 08:14:21.913623 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.913627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.913641 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:14:21.913684 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.913687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.913704 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.913787 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:21.913980 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.914799 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.915982 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.916502 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.916608 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.916671 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.916826 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.916868 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260619 08:14:21.917999 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.912517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41147 (local address 127.0.0.1:39238)
0619 08:14:21.913662 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.913674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.913742 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.914017 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.914018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.914020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:21.914070 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.914072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.915408 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.915409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.916625 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.916628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.917700 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.917701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.917779 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.917783 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.917857 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.917919 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"mutex_wait_us":56,"thread_start_us":56,"threads_started":1}
I20260619 08:14:21.918013 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.912736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.2.119.195:41147)
0619 08:14:21.912994 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.912999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.913743 (+ 744us) server_negotiation.cc:407] Connection header received
0619 08:14:21.913815 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.913817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.913834 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.913951 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:21.914135 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.915300 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.916762 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.917535 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.917606 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.917662 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.917876 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.917923 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260619 08:14:21.918015 3919 tablet_service.cc:1467] Processing CreateTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 (DEFAULT_TABLE table=test-table [id=4ffd29cea7d1449c8ce0060d134e1959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:21.918381 3919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73aecae6c14446cbcebfd3a2620f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:21.918861 4059 tablet_service.cc:1467] Processing CreateTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 (DEFAULT_TABLE table=test-table [id=4ffd29cea7d1449c8ce0060d134e1959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:21.919157 4059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73aecae6c14446cbcebfd3a2620f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:21.921609 4145 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Bootstrap starting.
I20260619 08:14:21.921659 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.912739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:37269 (local address 127.0.0.1:43748)
0619 08:14:21.913825 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.913835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.913880 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.914027 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.914028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.914029 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:21.914077 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.914080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.915409 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.915409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.916625 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.916628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.921416 (+ 4788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.921417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.921488 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.921493 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.921545 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.921589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"mutex_wait_us":44,"thread_start_us":53,"threads_started":1}
I20260619 08:14:21.921970 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.912987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43748 (local address 127.2.119.193:37269)
0619 08:14:21.913140 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.913144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.913887 (+ 743us) server_negotiation.cc:407] Connection header received
0619 08:14:21.913939 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.913941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.913955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.914025 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:21.914160 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.915301 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.916728 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.917489 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.917550 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.921789 (+ 4239us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.921836 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.921884 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260619 08:14:21.922564 4145 tablet_bootstrap.cc:654] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:21.922544 3779 tablet_service.cc:1467] Processing CreateTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 (DEFAULT_TABLE table=test-table [id=4ffd29cea7d1449c8ce0060d134e1959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:21.922813 4145 log.cc:824] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:21.922871 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b73aecae6c14446cbcebfd3a2620f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:21.923326 4146 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Bootstrap starting.
I20260619 08:14:21.923534 4145 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: No bootstrap required, opened a new log
I20260619 08:14:21.923621 4145 ts_tablet_manager.cc:1397] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:21.924322 4146 tablet_bootstrap.cc:654] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:21.924717 4146 log.cc:824] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:21.925746 4145 raft_consensus.cc:348] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.925922 4145 raft_consensus.cc:374] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:21.925959 4145 raft_consensus.cc:729] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40175f2a7af9409a814a97227cdef9f7, State: Initialized, Role: FOLLOWER
I20260619 08:14:21.925998 4146 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: No bootstrap required, opened a new log
I20260619 08:14:21.926050 4146 ts_tablet_manager.cc:1397] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:14:21.926222 4149 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Bootstrap starting.
I20260619 08:14:21.926272 4145 consensus_queue.cc:260] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.926698 4145 ts_tablet_manager.cc:1428] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:21.926827 3985 heartbeater.cc:503] Master 127.2.119.254:45223 was elected leader, sending a full tablet report...
I20260619 08:14:21.926857 4149 tablet_bootstrap.cc:654] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:21.927150 4149 log.cc:824] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:21.927838 4149 tablet_bootstrap.cc:492] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: No bootstrap required, opened a new log
I20260619 08:14:21.927929 4149 ts_tablet_manager.cc:1397] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:21.928115 4146 raft_consensus.cc:348] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.928238 4146 raft_consensus.cc:374] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:21.928267 4146 raft_consensus.cc:729] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3d66669b25466ab5a4fb1ac18defa5, State: Initialized, Role: FOLLOWER
I20260619 08:14:21.928400 4146 consensus_queue.cc:260] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.928659 4146 ts_tablet_manager.cc:1428] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:21.928716 4125 heartbeater.cc:503] Master 127.2.119.254:45223 was elected leader, sending a full tablet report...
I20260619 08:14:21.929940 4149 raft_consensus.cc:348] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.930078 4149 raft_consensus.cc:374] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:21.930120 4149 raft_consensus.cc:729] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b78943e42b444118a35568a8dabe1cb, State: Initialized, Role: FOLLOWER
I20260619 08:14:21.930228 4149 consensus_queue.cc:260] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.930464 4149 ts_tablet_manager.cc:1428] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:14:21.930526 3845 heartbeater.cc:503] Master 127.2.119.254:45223 was elected leader, sending a full tablet report...
I20260619 08:14:21.966977 4152 raft_consensus.cc:482] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:21.967113 4152 raft_consensus.cc:504] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.967521 4152 leader_election.cc:288] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145), 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269)
I20260619 08:14:21.970475 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.967881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:39293 (local address 127.2.119.193:37269)
0619 08:14:21.967953 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.967955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.967968 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:14:21.967999 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.968000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.968010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.968068 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:21.968279 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.968766 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.969714 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.970100 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.970198 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.970207 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.970371 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.970412 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:14:21.970475 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.967801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:37269 (local address 127.2.119.195:39293)
0619 08:14:21.967845 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.967857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.967925 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.968084 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.968085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.968088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:21.968178 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.968180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.968851 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.968853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.969618 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.969620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.970196 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.970197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.970283 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.970286 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.970360 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.970405 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 08:14:21.970736 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.967878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:47089 (local address 127.2.119.194:43145)
0619 08:14:21.967958 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.967960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.968230 (+ 270us) server_negotiation.cc:407] Connection header received
0619 08:14:21.968293 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.968294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.968304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.968378 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:21.968533 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.969003 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.969967 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.970332 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.970426 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.970434 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.970611 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.970673 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 08:14:21.970840 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73aecae6c14446cbcebfd3a2620f2b3" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb" is_pre_election: true
I20260619 08:14:21.970964 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.967802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:43145 (local address 127.2.119.195:47089)
0619 08:14:21.968150 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:21.968160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:21.968238 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:21.968382 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:21.968383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:21.968384 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:21.968463 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.968464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.969089 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.969090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.969879 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:21.969880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.970426 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.970427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:21.970502 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.970505 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:21.970815 (+ 310us) client_negotiation.cc:240] Negotiation successful
0619 08:14:21.970856 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":179,"thread_start_us":122,"threads_started":1}
I20260619 08:14:21.971011 3799 raft_consensus.cc:2454] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 0.
I20260619 08:14:21.971333 4011 leader_election.cc:302] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b78943e42b444118a35568a8dabe1cb, 8e3d66669b25466ab5a4fb1ac18defa5; no voters:
I20260619 08:14:21.971311 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73aecae6c14446cbcebfd3a2620f2b3" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40175f2a7af9409a814a97227cdef9f7" is_pre_election: true
I20260619 08:14:21.971470 3939 raft_consensus.cc:2454] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 0.
I20260619 08:14:21.971534 4152 raft_consensus.cc:2781] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:21.971602 4152 raft_consensus.cc:482] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:21.971655 4152 raft_consensus.cc:3037] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Advancing to term 1
W20260619 08:14:21.971751 3986 tablet.cc:2369] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:21.972404 4152 raft_consensus.cc:504] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.972596 4152 leader_election.cc:288] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 election: Requested vote from peers 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145), 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269)
I20260619 08:14:21.972683 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73aecae6c14446cbcebfd3a2620f2b3" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb"
I20260619 08:14:21.972774 3799 raft_consensus.cc:3037] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:21.973397 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b73aecae6c14446cbcebfd3a2620f2b3" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40175f2a7af9409a814a97227cdef9f7"
I20260619 08:14:21.973454 3939 raft_consensus.cc:3037] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:21.973479 3799 raft_consensus.cc:2454] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 1.
I20260619 08:14:21.973663 4011 leader_election.cc:302] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b78943e42b444118a35568a8dabe1cb, 8e3d66669b25466ab5a4fb1ac18defa5; no voters:
I20260619 08:14:21.973779 4152 raft_consensus.cc:2781] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:21.973913 4152 raft_consensus.cc:686] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 LEADER]: Becoming Leader. State: Replica: 8e3d66669b25466ab5a4fb1ac18defa5, State: Running, Role: LEADER
I20260619 08:14:21.973974 4152 consensus_queue.cc:237] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } }
I20260619 08:14:21.974248 3939 raft_consensus.cc:2454] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 1.
I20260619 08:14:21.974829 3641 catalog_manager.cc:5495] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:21.992164 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:21.989393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39246 (local address 127.2.119.195:41147)
0619 08:14:21.989567 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:21.989569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:21.989658 (+ 89us) server_negotiation.cc:407] Connection header received
0619 08:14:21.989766 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:21.989767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:21.989786 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:21.989866 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:21.990058 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.990531 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.991319 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:21.991519 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:21.991605 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:21.991813 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:21.991911 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:21.991998 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:21.992043 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 08:14:21.992063 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260619 08:14:21.994239 3799 raft_consensus.cc:1261] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Refusing update from remote peer 8e3d66669b25466ab5a4fb1ac18defa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:21.994254 3939 raft_consensus.cc:1261] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Refusing update from remote peer 8e3d66669b25466ab5a4fb1ac18defa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:21.994601 4152 consensus_queue.cc:1035] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:21.994747 4155 consensus_queue.cc:1035] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:21.997439 4160 mvcc.cc:205] Tried to move back new op lower bound from 7298485706725527552 to 7298485706646028288. Current Snapshot: MvccSnapshot[applied={T|T < 7298485706725527552}]
I20260619 08:14:21.998066 4159 mvcc.cc:205] Tried to move back new op lower bound from 7298485706725527552 to 7298485706646028288. Current Snapshot: MvccSnapshot[applied={T|T < 7298485706725527552 or (T in {7298485706725527552})}]
I20260619 08:14:22.000936 3642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:22.001046 3642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:22.001811 3642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Sending DeleteTablet for 3 replicas of tablet b73aecae6c14446cbcebfd3a2620f2b3
I20260619 08:14:22.002146 3919 tablet_service.cc:1514] Processing DeleteTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:21 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55616
I20260619 08:14:22.002259 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:21 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:43748
I20260619 08:14:22.002264 4059 tablet_service.cc:1514] Processing DeleteTablet for tablet b73aecae6c14446cbcebfd3a2620f2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:21 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:39238
I20260619 08:14:22.002502 4170 tablet_replica.cc:330] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: stopping tablet replica
I20260619 08:14:22.002486 3642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:14:22.002599 4170 raft_consensus.cc:2229] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 LEADER]: Raft consensus shutting down.
W20260619 08:14:22.002635 3642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:22.002714 4170 raft_consensus.cc:2258] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.003067 4169 tablet_replica.cc:330] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: stopping tablet replica
I20260619 08:14:22.003127 4170 ts_tablet_manager.cc:1905] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.003167 4169 raft_consensus.cc:2229] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:22.003261 4169 raft_consensus.cc:2258] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.003947 3919 tablet_service.cc:1467] Processing CreateTablet for tablet 0961bac51c344dd88be1b8d13dc16cad (DEFAULT_TABLE table=test-table [id=f60ca1c2f3e849249c56196898db974a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:22.004081 3919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0961bac51c344dd88be1b8d13dc16cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.004226 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 0961bac51c344dd88be1b8d13dc16cad (DEFAULT_TABLE table=test-table [id=f60ca1c2f3e849249c56196898db974a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:22.004315 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0961bac51c344dd88be1b8d13dc16cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.004848 4170 ts_tablet_manager.cc:1918] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:22.004909 4170 log.cc:1196] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/b73aecae6c14446cbcebfd3a2620f2b3
I20260619 08:14:22.005152 4170 ts_tablet_manager.cc:1939] T b73aecae6c14446cbcebfd3a2620f2b3 P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting consensus metadata
I20260619 08:14:22.005401 4059 tablet_service.cc:1467] Processing CreateTablet for tablet 0961bac51c344dd88be1b8d13dc16cad (DEFAULT_TABLE table=test-table [id=f60ca1c2f3e849249c56196898db974a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:22.005496 4059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0961bac51c344dd88be1b8d13dc16cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.005502 4145 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Bootstrap starting.
I20260619 08:14:22.005527 3627 catalog_manager.cc:4841] TS 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195:41147): tablet b73aecae6c14446cbcebfd3a2620f2b3 (table test-table [id=4ffd29cea7d1449c8ce0060d134e1959]) successfully deleted
I20260619 08:14:22.005911 4145 tablet_bootstrap.cc:654] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.005920 4169 ts_tablet_manager.cc:1905] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.006465 4145 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: No bootstrap required, opened a new log
I20260619 08:14:22.006505 4145 ts_tablet_manager.cc:1397] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:22.006646 4145 raft_consensus.cc:348] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.006686 4146 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Bootstrap starting.
I20260619 08:14:22.006716 4145 raft_consensus.cc:374] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.006738 4145 raft_consensus.cc:729] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40175f2a7af9409a814a97227cdef9f7, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.006789 4145 consensus_queue.cc:260] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.006888 4145 ts_tablet_manager.cc:1428] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:22.007076 4146 tablet_bootstrap.cc:654] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.007644 4146 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: No bootstrap required, opened a new log
I20260619 08:14:22.007705 4146 ts_tablet_manager.cc:1397] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:22.007858 4169 ts_tablet_manager.cc:1918] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:22.007860 4146 raft_consensus.cc:348] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.007920 4146 raft_consensus.cc:374] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.007943 4169 log.cc:1196] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/b73aecae6c14446cbcebfd3a2620f2b3
I20260619 08:14:22.007948 4146 raft_consensus.cc:729] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3d66669b25466ab5a4fb1ac18defa5, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.007993 4146 consensus_queue.cc:260] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.008071 4146 ts_tablet_manager.cc:1428] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:22.008433 4149 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Bootstrap starting.
I20260619 08:14:22.008672 4168 tablet_replica.cc:330] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: stopping tablet replica
I20260619 08:14:22.008715 4169 ts_tablet_manager.cc:1939] T b73aecae6c14446cbcebfd3a2620f2b3 P 7b78943e42b444118a35568a8dabe1cb: Deleting consensus metadata
I20260619 08:14:22.008832 4149 tablet_bootstrap.cc:654] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.008858 4168 raft_consensus.cc:2229] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:22.009065 4168 raft_consensus.cc:2258] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.009657 3627 catalog_manager.cc:4841] TS 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269): tablet b73aecae6c14446cbcebfd3a2620f2b3 (table test-table [id=4ffd29cea7d1449c8ce0060d134e1959]) successfully deleted
I20260619 08:14:22.009898 4149 tablet_bootstrap.cc:492] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: No bootstrap required, opened a new log
I20260619 08:14:22.009953 4149 ts_tablet_manager.cc:1397] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:14:22.009932 4168 ts_tablet_manager.cc:1905] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.010083 4149 raft_consensus.cc:348] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.010159 4149 raft_consensus.cc:374] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.010179 4149 raft_consensus.cc:729] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b78943e42b444118a35568a8dabe1cb, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.010224 4149 consensus_queue.cc:260] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.010316 4149 ts_tablet_manager.cc:1428] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:22.011615 4168 ts_tablet_manager.cc:1918] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:22.011701 4168 log.cc:1196] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/b73aecae6c14446cbcebfd3a2620f2b3
I20260619 08:14:22.011937 4168 ts_tablet_manager.cc:1939] T b73aecae6c14446cbcebfd3a2620f2b3 P 40175f2a7af9409a814a97227cdef9f7: Deleting consensus metadata
I20260619 08:14:22.012331 3629 catalog_manager.cc:4841] TS 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145): tablet b73aecae6c14446cbcebfd3a2620f2b3 (table test-table [id=4ffd29cea7d1449c8ce0060d134e1959]) successfully deleted
W20260619 08:14:22.062306 3846 tablet.cc:2369] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:22.103058 4150 raft_consensus.cc:482] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:22.103207 4150 raft_consensus.cc:504] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.103654 4150 leader_election.cc:288] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269), 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195:41147)
I20260619 08:14:22.108505 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.104635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:38647 (local address 127.2.119.195:41147)
0619 08:14:22.104831 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.104833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.105009 (+ 176us) server_negotiation.cc:407] Connection header received
0619 08:14:22.105033 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.105034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.105043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.105114 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:22.105323 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.106059 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.107577 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.108120 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.108162 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.108171 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.108375 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.108414 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260619 08:14:22.108788 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.103882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:37269 (local address 127.2.119.194:43553)
0619 08:14:22.103960 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.103971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.104034 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.104869 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.104870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.104872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:22.104948 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.104950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.106777 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.106778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.107794 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.107796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.108527 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.108528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.108607 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.108610 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.108664 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.108710 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 08:14:22.109066 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.104611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:43553 (local address 127.2.119.193:37269)
0619 08:14:22.104660 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.104663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.104673 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:22.104703 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.104704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.104714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.104771 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:22.105164 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.106682 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.107878 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.108429 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.108490 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.108497 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.108955 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.109000 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:22.109318 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.104249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41147 (local address 127.2.119.194:38647)
0619 08:14:22.104836 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.104846 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.104908 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.105115 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.105116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.105117 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:22.105192 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.105194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.106165 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.106166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.107476 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.107479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.108225 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.108226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.108298 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.108302 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.109052 (+ 750us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.109102 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":49,"threads_started":1}
I20260619 08:14:22.109248 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0961bac51c344dd88be1b8d13dc16cad" candidate_uuid: "40175f2a7af9409a814a97227cdef9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb" is_pre_election: true
I20260619 08:14:22.109369 3799 raft_consensus.cc:2454] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40175f2a7af9409a814a97227cdef9f7 in term 0.
I20260619 08:14:22.109562 3871 leader_election.cc:302] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40175f2a7af9409a814a97227cdef9f7, 7b78943e42b444118a35568a8dabe1cb; no voters:
I20260619 08:14:22.109750 4150 raft_consensus.cc:2781] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:22.109781 4150 raft_consensus.cc:482] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:22.109802 4150 raft_consensus.cc:3037] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.110637 4150 raft_consensus.cc:504] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.110790 4150 leader_election.cc:288] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269), 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195:41147)
I20260619 08:14:22.110970 4079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0961bac51c344dd88be1b8d13dc16cad" candidate_uuid: "40175f2a7af9409a814a97227cdef9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" is_pre_election: true
I20260619 08:14:22.111109 4079 raft_consensus.cc:2454] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40175f2a7af9409a814a97227cdef9f7 in term 0.
I20260619 08:14:22.111133 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0961bac51c344dd88be1b8d13dc16cad" candidate_uuid: "40175f2a7af9409a814a97227cdef9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb"
I20260619 08:14:22.111125 4078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0961bac51c344dd88be1b8d13dc16cad" candidate_uuid: "40175f2a7af9409a814a97227cdef9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3d66669b25466ab5a4fb1ac18defa5"
I20260619 08:14:22.111198 3799 raft_consensus.cc:3037] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.111209 4078 raft_consensus.cc:3037] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.111805 3799 raft_consensus.cc:2454] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40175f2a7af9409a814a97227cdef9f7 in term 1.
I20260619 08:14:22.111994 3871 leader_election.cc:302] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40175f2a7af9409a814a97227cdef9f7, 7b78943e42b444118a35568a8dabe1cb; no voters:
I20260619 08:14:22.112048 4078 raft_consensus.cc:2454] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40175f2a7af9409a814a97227cdef9f7 in term 1.
I20260619 08:14:22.112147 4150 raft_consensus.cc:2781] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:22.112257 4150 raft_consensus.cc:686] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 1 LEADER]: Becoming Leader. State: Replica: 40175f2a7af9409a814a97227cdef9f7, State: Running, Role: LEADER
I20260619 08:14:22.112326 4150 consensus_queue.cc:237] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.113044 3642 catalog_manager.cc:5495] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "40175f2a7af9409a814a97227cdef9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } health_report { overall_health: UNKNOWN } } }
W20260619 08:14:22.123817 4126 tablet.cc:2369] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 08:14:22 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:22.158519 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.150291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45223 (local address 127.0.0.1:55322)
0619 08:14:22.150453 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.150465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.150531 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.150682 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.150683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.150974 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:22.151029 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.151031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.152056 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.152057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.152569 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.152571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.153105 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.153106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.153157 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.153798 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:22.153803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:22.155159 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:22.157228 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:22.157235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:22.157237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:22.157697 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:22.157909 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:22.157910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:22.157910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:22.157967 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:22.158249 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:22.158254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:22.158302 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.158359 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.158445 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260619 08:14:22.158675 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.150362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.2.119.254:45223)
0619 08:14:22.150430 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.150433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.150535 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:14:22.150572 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.150573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.150585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.150661 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:22.151097 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.151974 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.152647 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.153003 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.153078 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.155248 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:22.155249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:22.155250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:22.155257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:22.157136 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.157769 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.157771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.157771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.157838 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.158029 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.158030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.158030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.158182 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:22.158256 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.158414 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.158514 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":43}
I20260619 08:14:22.161077 3642 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55322:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:14:22.161373 3642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:22.162995 3919 tablet_service.cc:1467] Processing CreateTablet for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2308c62938ca401f809545426a45919e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:22.163015 4059 tablet_service.cc:1467] Processing CreateTablet for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2308c62938ca401f809545426a45919e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:22.163180 3919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.163187 4059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.164611 4145 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: Bootstrap starting.
I20260619 08:14:22.164973 4146 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: Bootstrap starting.
I20260619 08:14:22.165026 4145 tablet_bootstrap.cc:654] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.165050 3779 tablet_service.cc:1467] Processing CreateTablet for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2308c62938ca401f809545426a45919e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:22.165186 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b99a3c7eb2f4388b64ce5cb2058d3bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:22.165338 4146 tablet_bootstrap.cc:654] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.165771 4145 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: No bootstrap required, opened a new log
I20260619 08:14:22.165835 4145 ts_tablet_manager.cc:1397] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:22.166003 4145 raft_consensus.cc:348] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.166108 4145 raft_consensus.cc:374] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.166133 4145 raft_consensus.cc:729] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40175f2a7af9409a814a97227cdef9f7, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.166175 4145 consensus_queue.cc:260] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.166293 4145 ts_tablet_manager.cc:1428] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:22.166553 4150 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.166627 4146 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: No bootstrap required, opened a new log
I20260619 08:14:22.166651 4150 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.166687 4146 ts_tablet_manager.cc:1397] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:14:22.166824 4149 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: Bootstrap starting.
I20260619 08:14:22.166802 4146 raft_consensus.cc:348] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.166865 4146 raft_consensus.cc:374] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.166885 4146 raft_consensus.cc:729] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3d66669b25466ab5a4fb1ac18defa5, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.166922 4146 consensus_queue.cc:260] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.167016 4146 ts_tablet_manager.cc:1428] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:22.167117 4164 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.167186 4164 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.167235 4149 tablet_bootstrap.cc:654] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:22.167869 4149 tablet_bootstrap.cc:492] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: No bootstrap required, opened a new log
I20260619 08:14:22.167912 4149 ts_tablet_manager.cc:1397] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:22.168066 4149 raft_consensus.cc:348] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.168119 4149 raft_consensus.cc:374] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:22.168133 4149 raft_consensus.cc:729] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b78943e42b444118a35568a8dabe1cb, State: Initialized, Role: FOLLOWER
I20260619 08:14:22.168183 4149 consensus_queue.cc:260] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.168272 4149 ts_tablet_manager.cc:1428] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:22.168375 4153 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.168529 4153 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.317638 4164 raft_consensus.cc:482] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:22.317746 4164 raft_consensus.cc:504] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.317966 4164 leader_election.cc:288] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269), 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145)
I20260619 08:14:22.318220 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b99a3c7eb2f4388b64ce5cb2058d3bd" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40175f2a7af9409a814a97227cdef9f7" is_pre_election: true
I20260619 08:14:22.318225 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b99a3c7eb2f4388b64ce5cb2058d3bd" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb" is_pre_election: true
I20260619 08:14:22.318318 3799 raft_consensus.cc:2454] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 0.
I20260619 08:14:22.318326 3939 raft_consensus.cc:2454] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 0.
I20260619 08:14:22.318493 4011 leader_election.cc:302] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b78943e42b444118a35568a8dabe1cb, 8e3d66669b25466ab5a4fb1ac18defa5; no voters:
I20260619 08:14:22.318606 4164 raft_consensus.cc:2781] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:22.318655 4164 raft_consensus.cc:482] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:22.318681 4164 raft_consensus.cc:3037] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.319520 4164 raft_consensus.cc:504] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.319657 4164 leader_election.cc:288] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 election: Requested vote from peers 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269), 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145)
I20260619 08:14:22.319846 3799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b99a3c7eb2f4388b64ce5cb2058d3bd" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b78943e42b444118a35568a8dabe1cb"
I20260619 08:14:22.319847 3939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b99a3c7eb2f4388b64ce5cb2058d3bd" candidate_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40175f2a7af9409a814a97227cdef9f7"
I20260619 08:14:22.319932 3939 raft_consensus.cc:3037] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.319932 3799 raft_consensus.cc:3037] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:22.320677 3799 raft_consensus.cc:2454] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 1.
I20260619 08:14:22.320677 3939 raft_consensus.cc:2454] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e3d66669b25466ab5a4fb1ac18defa5 in term 1.
I20260619 08:14:22.320851 4013 leader_election.cc:302] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40175f2a7af9409a814a97227cdef9f7, 8e3d66669b25466ab5a4fb1ac18defa5; no voters:
I20260619 08:14:22.320969 4164 raft_consensus.cc:2781] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:22.321054 4164 raft_consensus.cc:686] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 LEADER]: Becoming Leader. State: Replica: 8e3d66669b25466ab5a4fb1ac18defa5, State: Running, Role: LEADER
I20260619 08:14:22.321116 4164 consensus_queue.cc:237] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } }
I20260619 08:14:22.321376 4152 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: TxnStatusTablet state changed. Reason: New leader 8e3d66669b25466ab5a4fb1ac18defa5. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.321467 4152 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:22.321681 4187 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:14:22.321790 4187 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:14:22.321709 3642 catalog_manager.cc:5495] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } health_report { overall_health: HEALTHY } } }
I20260619 08:14:22.349048 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.345915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39262 (local address 127.2.119.195:41147)
0619 08:14:22.345985 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.345988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.346000 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:14:22.346084 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.346085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.346097 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.346174 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:22.346399 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.347067 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.348185 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.348401 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.348467 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.348725 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.348777 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.348838 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.348929 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.348948 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:14:22.349056 4182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.345814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41147 (local address 127.0.0.1:39262)
0619 08:14:22.345938 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.345950 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.345997 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.346228 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.346229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.346231 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:22.346304 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.346307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.347143 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.347144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.348081 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.348083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.348484 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.348485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.348557 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.348575 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.348860 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.348868 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.348924 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.348974 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 08:14:22.350111 3939 raft_consensus.cc:1261] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Refusing update from remote peer 8e3d66669b25466ab5a4fb1ac18defa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:22.350123 3799 raft_consensus.cc:1261] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Refusing update from remote peer 8e3d66669b25466ab5a4fb1ac18defa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:22.350324 4164 consensus_queue.cc:1035] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:22.350462 4152 consensus_queue.cc:1035] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:22.350925 4153 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: TxnStatusTablet state changed. Reason: New leader 8e3d66669b25466ab5a4fb1ac18defa5. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.351019 4153 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.350996 4150 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: TxnStatusTablet state changed. Reason: New leader 8e3d66669b25466ab5a4fb1ac18defa5. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.351076 4150 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.351717 4164 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.351796 4164 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:22.352154 4158 mvcc.cc:205] Tried to move back new op lower bound from 7298485708184248320 to 7298485708067475456. Current Snapshot: MvccSnapshot[applied={T|T < 7298485708184248320 or (T in {7298485708184248320})}]
I20260619 08:14:22.352312 4155 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.352388 4155 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 8e3d66669b25466ab5a4fb1ac18defa5: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:22.352775 4153 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.352854 4153 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 7b78943e42b444118a35568a8dabe1cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.352800 4150 tablet_replica.cc:439] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 } } peers { permanent_uuid: "40175f2a7af9409a814a97227cdef9f7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43145 } } peers { permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 } } }
I20260619 08:14:22.352880 4150 tablet_replica.cc:442] T 5b99a3c7eb2f4388b64ce5cb2058d3bd P 40175f2a7af9409a814a97227cdef9f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:22.356515 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.354136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.2.119.194:43145)
0619 08:14:22.354178 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.354181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.354190 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:22.354276 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.354277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.354295 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.354383 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:22.354482 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.354963 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.355679 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.355889 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.355989 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.356154 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.356290 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.356373 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.356418 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.356445 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:14:22.361464 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.358875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:57607 (local address 127.2.119.195:41147)
0619 08:14:22.358920 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.358922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.359018 (+ 96us) server_negotiation.cc:407] Connection header received
0619 08:14:22.359094 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.359095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.359106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.359172 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:22.359368 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.359802 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.360700 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.360898 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.360980 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.361157 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.361195 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.361272 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.361378 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.361398 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:22.361501 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.358793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41147 (local address 127.2.119.194:57607)
0619 08:14:22.358946 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.358956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.359015 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.359181 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.359182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.359183 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:22.359231 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.359232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.359936 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.359937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.360609 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.360611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.360983 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.360984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.361053 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.361066 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.361324 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.361327 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.361373 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.361417 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260619 08:14:22.363785 3799 raft_consensus.cc:1261] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Refusing update from remote peer 40175f2a7af9409a814a97227cdef9f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:22.363806 4078 raft_consensus.cc:1261] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Refusing update from remote peer 40175f2a7af9409a814a97227cdef9f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:22.364048 4175 consensus_queue.cc:1035] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b78943e42b444118a35568a8dabe1cb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:22.364166 4150 consensus_queue.cc:1035] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e3d66669b25466ab5a4fb1ac18defa5" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:22.373054 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.370361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:46805 (local address 127.2.119.194:43145)
0619 08:14:22.370534 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.370535 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.370544 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:22.370567 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.370568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.370576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.370614 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:22.370758 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.371191 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.372375 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.372640 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.372689 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.372866 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.372889 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.372922 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.372991 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.373006 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260619 08:14:22.373102 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.370288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:43145 (local address 127.2.119.195:46805)
0619 08:14:22.370417 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:22.370429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:22.370468 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:22.370645 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:22.370646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:22.370647 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:22.370691 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.370692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.371296 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.371297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.372277 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:22.372280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.372708 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.372709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:22.372773 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.372788 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:22.372938 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:22.372939 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:22.372991 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:14:22.373030 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260619 08:14:22.382797 3640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:22.382913 3640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:22.383540 3640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 417d64089e4346e9be9f3121a3051627: Sending DeleteTablet for 3 replicas of tablet 0961bac51c344dd88be1b8d13dc16cad
I20260619 08:14:22.383776 3919 tablet_service.cc:1514] Processing DeleteTablet for tablet 0961bac51c344dd88be1b8d13dc16cad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:22 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55616
I20260619 08:14:22.383874 4168 tablet_replica.cc:330] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: stopping tablet replica
I20260619 08:14:22.383857 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet 0961bac51c344dd88be1b8d13dc16cad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:22 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:43748
I20260619 08:14:22.383924 4168 raft_consensus.cc:2229] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:22.384047 4168 raft_consensus.cc:2258] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.384052 4059 tablet_service.cc:1514] Processing DeleteTablet for tablet 0961bac51c344dd88be1b8d13dc16cad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:22 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:39238
I20260619 08:14:22.384127 4169 tablet_replica.cc:330] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: stopping tablet replica
I20260619 08:14:22.384176 4169 raft_consensus.cc:2229] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:22.384212 4169 raft_consensus.cc:2258] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.384330 4170 tablet_replica.cc:330] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: stopping tablet replica
I20260619 08:14:22.384384 4170 raft_consensus.cc:2229] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:22.384419 4170 raft_consensus.cc:2258] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:22.384842 4168 ts_tablet_manager.cc:1905] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.384836 4169 ts_tablet_manager.cc:1905] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.386546 4170 ts_tablet_manager.cc:1905] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:22.386620 4168 ts_tablet_manager.cc:1918] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:22.386689 4168 log.cc:1196] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/0961bac51c344dd88be1b8d13dc16cad
I20260619 08:14:22.386929 4168 ts_tablet_manager.cc:1939] T 0961bac51c344dd88be1b8d13dc16cad P 40175f2a7af9409a814a97227cdef9f7: Deleting consensus metadata
I20260619 08:14:22.387230 3629 catalog_manager.cc:4841] TS 40175f2a7af9409a814a97227cdef9f7 (127.2.119.194:43145): tablet 0961bac51c344dd88be1b8d13dc16cad (table test-table [id=f60ca1c2f3e849249c56196898db974a]) successfully deleted
I20260619 08:14:22.387976 4170 ts_tablet_manager.cc:1918] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:22.388048 4170 log.cc:1196] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/0961bac51c344dd88be1b8d13dc16cad
I20260619 08:14:22.388254 4170 ts_tablet_manager.cc:1939] T 0961bac51c344dd88be1b8d13dc16cad P 8e3d66669b25466ab5a4fb1ac18defa5: Deleting consensus metadata
I20260619 08:14:22.388557 4169 ts_tablet_manager.cc:1918] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:22.388563 3627 catalog_manager.cc:4841] TS 8e3d66669b25466ab5a4fb1ac18defa5 (127.2.119.195:41147): tablet 0961bac51c344dd88be1b8d13dc16cad (table test-table [id=f60ca1c2f3e849249c56196898db974a]) successfully deleted
I20260619 08:14:22.388614 4169 log.cc:1196] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/0961bac51c344dd88be1b8d13dc16cad
I20260619 08:14:22.388808 4169 ts_tablet_manager.cc:1939] T 0961bac51c344dd88be1b8d13dc16cad P 7b78943e42b444118a35568a8dabe1cb: Deleting consensus metadata
I20260619 08:14:22.389123 3627 catalog_manager.cc:4841] TS 7b78943e42b444118a35568a8dabe1cb (127.2.119.193:37269): tablet 0961bac51c344dd88be1b8d13dc16cad (table test-table [id=f60ca1c2f3e849249c56196898db974a]) successfully deleted
Jun 19 08:14:22 dist-test-slave-d0b5 krb5kdc[3597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:22.397274 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.389266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.2.119.193:37269)
0619 08:14:22.389324 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.389326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.389539 (+ 213us) server_negotiation.cc:407] Connection header received
0619 08:14:22.389612 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.389613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.389623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.389678 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:22.390231 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.390704 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.391434 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.391638 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.391715 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.394493 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:22.394502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:22.394503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:22.394518 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:22.396089 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.396451 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.396454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.396455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.396516 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.396721 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.396722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.396722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.396876 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:22.397029 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.397096 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.397174 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:14:22.397735 3839 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43764
I20260619 08:14:22.404611 3692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:22.398663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.2.119.254:45223)
0619 08:14:22.398707 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:22.398709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:22.398772 (+ 63us) server_negotiation.cc:407] Connection header received
0619 08:14:22.398852 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:22.398852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:22.398863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:22.398910 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:22.399474 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.400042 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.400708 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:22.400907 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:22.400977 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:22.402107 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:22.402108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:22.402108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:22.402116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:22.403491 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.403850 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.403851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.403852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.403910 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:22.404107 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:22.404108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:22.404108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:22.404220 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:22.404335 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:22.404430 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:14:22.404502 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:14:22.404807 3640 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55332
I20260619 08:14:22.405339 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3714
I20260619 08:14:22.411191 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3854
I20260619 08:14:22.416430 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3994
I20260619 08:14:22.422375 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 3610
2026-06-19T08:14:22Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2383 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:22 dist-test-slave-d0b5 krb5kdc[4207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:22 dist-test-slave-d0b5 krb5kdc[4207](info): set up 2 sockets
Jun 19 08:14:22 dist-test-slave-d0b5 krb5kdc[4207](info): commencing operation
krb5kdc: starting...
W20260619 08:14:23.686121 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:23 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:23.809044 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42723
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:42723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:23.890807 4220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:23.891017 4220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:23.891040 4220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:23.892273 4220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:23.892314 4220 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:14:23.892328 4220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:23.892365 4220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:23.892392 4220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:23.892408 4220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:23.893947 4220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:42723
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:23.894200 4220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:23.894438 4220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:23.899235 4226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:23.899250 4225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:23.899291 4228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:23.899534 4220 server_base.cc:1034] running on GCE node
I20260619 08:14:23.899729 4220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:23.899958 4220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:23.901095 4220 hybrid_clock.cc:648] HybridClock initialized: now 1781856863901081 us; error 29 us; skew 500 ppm
Jun 19 08:14:23 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856863, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:23.903478 4220 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:23.903904 4220 webserver.cc:460] Webserver started at http://127.2.119.254:44063/ using document root <none> and password file <none>
I20260619 08:14:23.904134 4220 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:23.904178 4220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:23.904320 4220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:23.905464 4220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "c08d1393123243c695ff4eda38864165"
format_stamp: "Formatted at 2026-06-19 08:14:23 on dist-test-slave-d0b5"
server_key: "ae42a192cf236aff5b77c8d9ebb04d4e"
server_key_iv: "f1951ac3ef14808675b46ac3a7bb1010"
server_key_version: "encryptionkey@0"
I20260619 08:14:23.905818 4220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "c08d1393123243c695ff4eda38864165"
format_stamp: "Formatted at 2026-06-19 08:14:23 on dist-test-slave-d0b5"
server_key: "ae42a192cf236aff5b77c8d9ebb04d4e"
server_key_iv: "f1951ac3ef14808675b46ac3a7bb1010"
server_key_version: "encryptionkey@0"
I20260619 08:14:23.907138 4220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:14:23.907910 4235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:23.908087 4220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:23.908171 4220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "c08d1393123243c695ff4eda38864165"
format_stamp: "Formatted at 2026-06-19 08:14:23 on dist-test-slave-d0b5"
server_key: "ae42a192cf236aff5b77c8d9ebb04d4e"
server_key_iv: "f1951ac3ef14808675b46ac3a7bb1010"
server_key_version: "encryptionkey@0"
I20260619 08:14:23.908250 4220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:23.921494 4220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:23.923619 4220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:23.923763 4220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:23.928395 4220 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:42723
I20260619 08:14:23.928480 4297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:42723 every 8 connection(s)
I20260619 08:14:23.928800 4220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:23.929541 4298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:23.931854 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Bootstrap starting.
I20260619 08:14:23.932363 4298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:23.932590 4298 log.cc:824] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:23.933256 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: No bootstrap required, opened a new log
I20260619 08:14:23.933485 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 4220
I20260619 08:14:23.933563 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:23.933735 2527 external_mini_cluster.cc:1428] Setting key 84688bb8e50940d5715de2f3c19a6764
I20260619 08:14:23.936055 4298 raft_consensus.cc:348] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } }
I20260619 08:14:23.936226 4298 raft_consensus.cc:374] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:23.936264 4298 raft_consensus.cc:729] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08d1393123243c695ff4eda38864165, State: Initialized, Role: FOLLOWER
I20260619 08:14:23.936383 4298 consensus_queue.cc:260] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } }
I20260619 08:14:23.936465 4298 raft_consensus.cc:388] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:23.936486 4298 raft_consensus.cc:482] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:23.936553 4298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:23.937150 4298 raft_consensus.cc:504] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } }
I20260619 08:14:23.937250 4298 leader_election.cc:302] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c08d1393123243c695ff4eda38864165; no voters:
I20260619 08:14:23.937424 4298 leader_election.cc:288] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:23.937494 4303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:23.937656 4298 sys_catalog.cc:564] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:23.937664 4303 raft_consensus.cc:686] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [term 1 LEADER]: Becoming Leader. State: Replica: c08d1393123243c695ff4eda38864165, State: Running, Role: LEADER
I20260619 08:14:23.937786 4303 consensus_queue.cc:237] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } }
I20260619 08:14:23.938120 4303 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c08d1393123243c695ff4eda38864165. Latest consensus state: current_term: 1 leader_uuid: "c08d1393123243c695ff4eda38864165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } } }
I20260619 08:14:23.938184 4303 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:23.938297 4305 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c08d1393123243c695ff4eda38864165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08d1393123243c695ff4eda38864165" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42723 } } }
I20260619 08:14:23.938370 4305 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:23.938365 4308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:23.939276 4308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:23.941026 4308 catalog_manager.cc:1347] Generated new cluster ID: 269967aa40e84ddfa2b5e803ba90fb00
I20260619 08:14:23.941076 4308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:23 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:23.946640 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:23.934247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.2.119.254:42723)
0619 08:14:23.934456 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:23.934465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:23.934498 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:14:23.936009 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:23.936018 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:23.936082 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:23.936227 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:23.936982 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:23.938431 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:23.939553 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:23.940151 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:23.940256 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:23.943010 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:23.943014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:23.943014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:23.943026 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:23.945019 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:23.945435 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:23.945437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:23.945438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:23.945545 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:23.945741 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:23.945742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:23.945742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:23.946050 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:23.946213 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:23.946289 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 08:14:23.946440 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260619 08:14:23.950134 4308 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:23.950812 4308 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:23.960898 4308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Generated new TSK 0
I20260619 08:14:23.961105 4308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:23.994760 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42723
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:24.079072 4324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:24.079344 4324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:24.079370 4324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:24.080534 4324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:24.080579 4324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:24.080601 4324 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:14:24.080614 4324 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:24.080659 4324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:24.082221 4324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:24.082484 4324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:24.082736 4324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:24.087922 4329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:24.087908 4330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:24.087908 4332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:24.088157 4324 server_base.cc:1034] running on GCE node
I20260619 08:14:24.088356 4324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:24.088624 4324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:24.089790 4324 hybrid_clock.cc:648] HybridClock initialized: now 1781856864089775 us; error 28 us; skew 500 ppm
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:24.092288 4324 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:24.092756 4324 webserver.cc:460] Webserver started at http://127.2.119.193:40055/ using document root <none> and password file <none>
I20260619 08:14:24.092995 4324 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:24.093066 4324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:24.093225 4324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:24.094333 4324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "094b609bfe914d6ca04bad6771d876fb"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "4d3b4c31e5ae61bb333ce94fb94ec0b4"
server_key_iv: "68f04aedcb3cec9d1ad3275739d62c3d"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.094719 4324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "094b609bfe914d6ca04bad6771d876fb"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "4d3b4c31e5ae61bb333ce94fb94ec0b4"
server_key_iv: "68f04aedcb3cec9d1ad3275739d62c3d"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.096180 4324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:24.096916 4339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.097111 4324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:24.097223 4324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "094b609bfe914d6ca04bad6771d876fb"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "4d3b4c31e5ae61bb333ce94fb94ec0b4"
server_key_iv: "68f04aedcb3cec9d1ad3275739d62c3d"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.097301 4324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:24.105743 4324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:24.107585 4324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:24.107749 4324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:24.108287 4324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:24.108345 4324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.108383 4324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:24.108403 4324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:24.115682 4324 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:43291
I20260619 08:14:24.115765 4454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:43291 every 8 connection(s)
I20260619 08:14:24.116109 4324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:24.118574 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.108779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:59221 (local address 127.2.119.254:42723)
0619 08:14:24.108847 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.108849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.109105 (+ 256us) server_negotiation.cc:407] Connection header received
0619 08:14:24.109899 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.109901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.109911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.109972 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.110991 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.111520 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.112698 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.112927 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.113000 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.115592 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.115593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.115594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.115601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.117176 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.117616 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.117617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.117618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.117679 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.117934 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.117934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.117935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.118049 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.118119 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.118396 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.118483 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:24.118733 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.108756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.193:59221)
0619 08:14:24.108960 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.109010 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.109781 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.110049 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.110054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.110474 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.110787 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.110820 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.111625 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.111627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.112608 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.112610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.113032 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.113033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.113157 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.113671 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.113680 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.115496 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.117272 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.117279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.117281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.117525 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.117746 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.117747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.117748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.117861 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.118121 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.118128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.118261 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.118364 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.118516 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260619 08:14:24.119213 4455 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42723
I20260619 08:14:24.119300 4455 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:24.119482 4455 heartbeater.cc:511] Master 127.2.119.254:42723 requested a full tablet report, sending...
I20260619 08:14:24.119867 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 4324
I20260619 08:14:24.119995 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:24.120205 2527 external_mini_cluster.cc:1428] Setting key 6711661bcf844b911916c3659364ea9e
I20260619 08:14:24.120414 4252 ts_manager.cc:194] Registered new tserver with Master: 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291)
I20260619 08:14:24.121352 4252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:59221
I20260619 08:14:24.126559 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.120257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:43113 (local address 127.2.119.254:42723)
0619 08:14:24.120291 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.120293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.120556 (+ 263us) server_negotiation.cc:407] Connection header received
0619 08:14:24.120644 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.120644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.120653 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.120681 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.121081 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.121824 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.122597 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.122913 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.122991 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.123978 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.123979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.123979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.123985 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.125456 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.125844 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.125845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.125845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.125892 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.126121 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.126121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.126122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.126214 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.126262 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.126450 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.126502 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:14:24.126701 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.120222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.193:43113)
0619 08:14:24.120497 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.120508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.120579 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.120737 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.120737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.120928 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.121005 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.121007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.121921 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.121922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.122515 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.122517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.122993 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.122994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.123051 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.123475 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.123481 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.123896 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.125532 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.125534 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.125534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.125771 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.125979 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.125980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.125980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.126054 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.126276 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.126278 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.126347 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.126436 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.126548 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":129,"threads_started":1}
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:24.148952 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42723
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:24.228055 4464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:24.228252 4464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:24.228277 4464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:24.229374 4464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:24.229409 4464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:24.229429 4464 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:14:24.229442 4464 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:24.229458 4464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:24.230819 4464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:24.231002 4464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:24.231171 4464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:24.236219 4472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:24.236258 4469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:24.236232 4470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:24.236541 4464 server_base.cc:1034] running on GCE node
I20260619 08:14:24.236713 4464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:24.236958 4464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:24.238103 4464 hybrid_clock.cc:648] HybridClock initialized: now 1781856864238066 us; error 52 us; skew 500 ppm
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:24.240759 4464 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:24.241235 4464 webserver.cc:460] Webserver started at http://127.2.119.194:40761/ using document root <none> and password file <none>
I20260619 08:14:24.241508 4464 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:24.241559 4464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:24.241706 4464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:24.242758 4464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "ea33030155ed4a18bff9192342c9e6c4"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "714daab6f7619a1cc8afbd2f88a48ab2"
server_key_iv: "d322f1689fa154d77fed36b60b5b0a5e"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.243127 4464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "ea33030155ed4a18bff9192342c9e6c4"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "714daab6f7619a1cc8afbd2f88a48ab2"
server_key_iv: "d322f1689fa154d77fed36b60b5b0a5e"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.244521 4464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:24.245376 4479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.245566 4464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:24.245664 4464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "ea33030155ed4a18bff9192342c9e6c4"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "714daab6f7619a1cc8afbd2f88a48ab2"
server_key_iv: "d322f1689fa154d77fed36b60b5b0a5e"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.245751 4464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:24.258203 4464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:24.260087 4464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:24.260244 4464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:24.260816 4464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:24.260879 4464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.260912 4464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:24.260931 4464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:24.267938 4464 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:42297
I20260619 08:14:24.268277 4464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:24.268494 4594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:42297 every 8 connection(s)
I20260619 08:14:24.271168 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.261333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:50953 (local address 127.2.119.254:42723)
0619 08:14:24.261388 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.261391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.261582 (+ 191us) server_negotiation.cc:407] Connection header received
0619 08:14:24.262060 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.262062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.262074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.262156 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.262789 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.263639 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.265285 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.265611 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.265702 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.268409 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.268410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.268411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.268420 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.270022 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.270382 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.270383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.270383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.270432 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.270636 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.270637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.270637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.270734 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.270783 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.271048 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.271106 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:14:24.271360 4488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.261267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.194:50953)
0619 08:14:24.261450 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.261490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.261970 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.262210 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.262214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.262518 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.262713 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.262719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.263737 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.263738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.265177 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.265180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.265746 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.265747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.265924 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.266431 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.266443 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.268278 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.270091 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.270095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.270097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.270297 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.270487 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.270488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.270488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.270583 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.270787 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.270791 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.270936 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.271020 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.271134 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":118,"threads_started":1}
I20260619 08:14:24.271896 4595 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42723
I20260619 08:14:24.271981 4595 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:24.272161 4595 heartbeater.cc:511] Master 127.2.119.254:42723 requested a full tablet report, sending...
I20260619 08:14:24.272506 4252 ts_manager.cc:194] Registered new tserver with Master: ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297)
I20260619 08:14:24.272998 4252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:50953
I20260619 08:14:24.273514 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 4464
I20260619 08:14:24.273617 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:24.273823 2527 external_mini_cluster.cc:1428] Setting key 5b67809cdd4bb036e2859705a28ea098
I20260619 08:14:24.279345 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.272871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.194:54385)
0619 08:14:24.273006 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.273017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.273081 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.273214 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.273215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.273487 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.273569 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.273571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.274466 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.274467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.275281 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.275284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.275653 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.275654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.275734 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.276196 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.276200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.276459 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.278294 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.278295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.278295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.278494 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.278719 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.278720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.278720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.278790 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.279118 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.279119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.279152 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.279205 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.279278 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260619 08:14:24.279448 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.272958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:54385 (local address 127.2.119.254:42723)
0619 08:14:24.273004 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.273006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.273087 (+ 81us) server_negotiation.cc:407] Connection header received
0619 08:14:24.273142 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.273142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.273153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.273211 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.273671 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.274334 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.275388 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.275573 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.275650 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.276543 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.276544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.276545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.276553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.278184 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.278577 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.278578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.278578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.278641 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.278856 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.278857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.278857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.279047 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.279124 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.279236 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.279319 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:24.300575 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42723
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:24.381019 4604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:24.381218 4604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:24.381243 4604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:24.382393 4604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:24.382436 4604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:24.382457 4604 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 08:14:24.382468 4604 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:24.382485 4604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:24.383829 4604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:42723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:24.383996 4604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:24.384169 4604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:24.389003 4610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:24.389030 4604 server_base.cc:1034] running on GCE node
W20260619 08:14:24.388983 4609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:24.389107 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:24.389475 4604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:24.389779 4604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:24.390954 4604 hybrid_clock.cc:648] HybridClock initialized: now 1781856864390936 us; error 28 us; skew 500 ppm
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:24.393520 4604 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:24.393977 4604 webserver.cc:460] Webserver started at http://127.2.119.195:40433/ using document root <none> and password file <none>
I20260619 08:14:24.394196 4604 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:24.394259 4604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:24.394389 4604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:24.395466 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "09dce63eab6048ca93802984739cf055"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "2171b450b9b7509d644bc0f5282d5069"
server_key_iv: "31099d327e8e4fcb3d25debb9cc87bff"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.395828 4604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "09dce63eab6048ca93802984739cf055"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "2171b450b9b7509d644bc0f5282d5069"
server_key_iv: "31099d327e8e4fcb3d25debb9cc87bff"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.397114 4604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:24.397930 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.398108 4604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.398200 4604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "09dce63eab6048ca93802984739cf055"
format_stamp: "Formatted at 2026-06-19 08:14:24 on dist-test-slave-d0b5"
server_key: "2171b450b9b7509d644bc0f5282d5069"
server_key_iv: "31099d327e8e4fcb3d25debb9cc87bff"
server_key_version: "encryptionkey@0"
I20260619 08:14:24.398280 4604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:24.406490 4604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:24.408401 4604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:24.408546 4604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:24.409122 4604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:24.409171 4604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.409204 4604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:24.409227 4604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:24.416781 4604 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:37033
I20260619 08:14:24.416846 4734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:37033 every 8 connection(s)
I20260619 08:14:24.417213 4604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:24.418888 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.409509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:37533 (local address 127.2.119.254:42723)
0619 08:14:24.409571 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.409573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.409859 (+ 286us) server_negotiation.cc:407] Connection header received
0619 08:14:24.410600 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.410602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.410628 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.410703 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.411576 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.412142 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.413169 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.413440 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.413526 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.416155 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.416156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.416156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.416164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.417829 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.418176 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.418176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.418177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.418228 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.418402 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.418402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.418402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.418499 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.418547 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.418767 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.418824 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:24.419087 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.409509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.195:37533)
0619 08:14:24.409700 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.409748 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.410425 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.410796 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.410800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.411182 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.411487 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.411494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.412240 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.412241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.413070 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.413072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.413537 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.413538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.413673 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.414199 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.414209 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.416005 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.417900 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.417904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.417905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.418107 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.418268 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.418269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.418269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.418347 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.418555 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.418559 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.418664 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.418744 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.418854 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260619 08:14:24.419579 4735 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42723
I20260619 08:14:24.419682 4735 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:24.419848 4735 heartbeater.cc:511] Master 127.2.119.254:42723 requested a full tablet report, sending...
I20260619 08:14:24.420239 4251 ts_manager.cc:194] Registered new tserver with Master: 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:24.420799 4251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:37533
I20260619 08:14:24.425209 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 4604
I20260619 08:14:24.425367 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:24.425623 2527 external_mini_cluster.cc:1428] Setting key 0b5b9e7a939d7ab74e61eadf02077a43
I20260619 08:14:24.426316 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.420678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:36857 (local address 127.2.119.254:42723)
0619 08:14:24.420718 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.420720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.420793 (+ 73us) server_negotiation.cc:407] Connection header received
0619 08:14:24.420873 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.420874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.420884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.420938 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.421328 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.421808 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.422524 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.422730 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.422805 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.423592 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.423593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.423594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.423601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.424939 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.425491 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.425492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.425493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.425561 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.425799 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.425800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.425800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.425932 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.425997 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.426161 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.426240 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:14:24.426324 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.420584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.2.119.195:36857)
0619 08:14:24.420728 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.420739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.420799 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.420961 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.420962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.421162 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.421230 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.421231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.421909 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.421910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.422440 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.422442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.422803 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.422804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.422889 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.423225 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.423228 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.423490 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.425025 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.425026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.425027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.425247 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.425629 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.425630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.425631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.425700 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.426001 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.426002 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.426048 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.426118 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.426231 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260619 08:14:24.426541 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:24.449470 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.442341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.2.119.254:42723)
0619 08:14:24.442400 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.442402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.442431 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:14:24.442545 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.442545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.442553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.442642 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.443196 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.443687 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.444419 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.444609 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.444693 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.446987 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.446989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.446989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.446995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.448303 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.448693 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.448694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.448694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.448792 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.449011 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.449012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.449012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.449117 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.449194 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.449278 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.449407 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:14:24.451220 4251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:24.452775 4251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:24.459182 4753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.455195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43291 (local address 127.0.0.1:46698)
0619 08:14:24.455427 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.455437 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.455491 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.455737 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.455738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.455740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:24.455797 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.455798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.456731 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.456732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.457910 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.457913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.458903 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.458968 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.458972 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.459026 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.459086 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":33,"threads_started":1}
I20260619 08:14:24.459189 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.454963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:42297 (local address 127.0.0.1:55840)
0619 08:14:24.455277 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.455319 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.455382 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.455631 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.455633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.455635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:24.455699 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.455701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.457026 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.457027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.458134 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.458136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.458840 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.458931 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.458945 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.459045 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.459090 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":85,"threads_started":1}
I20260619 08:14:24.459347 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.454998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.0.0.1:47634)
0619 08:14:24.455277 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.455319 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.455382 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.455957 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.455957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.455960 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:24.455998 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.455999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.457210 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.457212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.458220 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.458223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.459120 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.459121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.459181 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.459183 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.459232 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.459274 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":53,"threads_started":1}
I20260619 08:14:24.459564 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.455316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.2.119.193:43291)
0619 08:14:24.455507 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.455511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.455521 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:24.455561 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.455562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.455578 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.455652 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:24.455881 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.456634 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458000 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.458739 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458810 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.458866 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.459422 (+ 556us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.459470 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
I20260619 08:14:24.459619 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.455098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.2.119.195:37033)
0619 08:14:24.455720 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.455730 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.455741 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:24.455782 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.455784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.455799 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.455869 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:24.456085 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.457125 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458290 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.458978 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.459039 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.459088 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.459475 (+ 387us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.459518 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":98,"threads_started":1}
I20260619 08:14:24.459200 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.455075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.2.119.194:42297)
0619 08:14:24.455360 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.455366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.455380 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:14:24.455515 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.455522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.455535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.455626 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:24.455781 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.456934 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458229 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.458736 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.458835 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.458892 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.459062 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.459100 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260619 08:14:24.460275 4389 tablet_service.cc:1467] Processing CreateTablet for tablet e6f984a71b964a1fa87769e186ecff3d (DEFAULT_TABLE table=test-table [id=c65c024c36244d13804470c01e7e8379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.460276 4529 tablet_service.cc:1467] Processing CreateTablet for tablet e6f984a71b964a1fa87769e186ecff3d (DEFAULT_TABLE table=test-table [id=c65c024c36244d13804470c01e7e8379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.460340 4669 tablet_service.cc:1467] Processing CreateTablet for tablet e6f984a71b964a1fa87769e186ecff3d (DEFAULT_TABLE table=test-table [id=c65c024c36244d13804470c01e7e8379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.460590 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f984a71b964a1fa87769e186ecff3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.460602 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f984a71b964a1fa87769e186ecff3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.460602 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6f984a71b964a1fa87769e186ecff3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.463227 4755 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Bootstrap starting.
I20260619 08:14:24.464088 4755 tablet_bootstrap.cc:654] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.464357 4757 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Bootstrap starting.
I20260619 08:14:24.464443 4756 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Bootstrap starting.
I20260619 08:14:24.464443 4755 log.cc:824] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:24.464996 4757 tablet_bootstrap.cc:654] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.465037 4756 tablet_bootstrap.cc:654] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.465268 4755 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: No bootstrap required, opened a new log
I20260619 08:14:24.465348 4757 log.cc:824] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:24.465376 4755 ts_tablet_manager.cc:1397] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:24.465378 4756 log.cc:824] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:24.466012 4757 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: No bootstrap required, opened a new log
I20260619 08:14:24.466091 4757 ts_tablet_manager.cc:1397] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:24.467094 4755 raft_consensus.cc:348] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.467234 4755 raft_consensus.cc:374] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.467258 4755 raft_consensus.cc:729] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dce63eab6048ca93802984739cf055, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.467368 4755 consensus_queue.cc:260] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.467626 4755 ts_tablet_manager.cc:1428] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:24.467659 4735 heartbeater.cc:503] Master 127.2.119.254:42723 was elected leader, sending a full tablet report...
I20260619 08:14:24.467646 4757 raft_consensus.cc:348] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.467806 4757 raft_consensus.cc:374] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.467844 4757 raft_consensus.cc:729] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.467969 4757 consensus_queue.cc:260] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.468245 4757 ts_tablet_manager.cc:1428] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:24.468284 4756 tablet_bootstrap.cc:492] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: No bootstrap required, opened a new log
I20260619 08:14:24.468282 4455 heartbeater.cc:503] Master 127.2.119.254:42723 was elected leader, sending a full tablet report...
I20260619 08:14:24.468338 4756 ts_tablet_manager.cc:1397] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 08:14:24.470090 4756 raft_consensus.cc:348] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.470214 4756 raft_consensus.cc:374] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.470235 4756 raft_consensus.cc:729] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.470338 4756 consensus_queue.cc:260] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.470626 4756 ts_tablet_manager.cc:1428] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 08:14:24.470664 4595 heartbeater.cc:503] Master 127.2.119.254:42723 was elected leader, sending a full tablet report...
W20260619 08:14:24.518982 4596 tablet.cc:2369] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:24.597092 4761 raft_consensus.cc:482] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:24.597222 4761 raft_consensus.cc:504] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.597622 4761 leader_election.cc:288] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297), 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291)
I20260619 08:14:24.600821 4628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.597856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:42297 (local address 127.2.119.195:39135)
0619 08:14:24.597920 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.597933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.597997 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.598161 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.598161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.598163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:24.598260 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.598262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.598925 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.598926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.600007 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.600010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.600580 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.600664 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.600667 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.600712 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.600752 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:14:24.600849 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.597971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:39135 (local address 127.2.119.194:42297)
0619 08:14:24.598032 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.598035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.598047 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:14:24.598079 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.598079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.598090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.598150 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:24.598344 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.598845 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600084 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.600492 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600549 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.600557 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.600741 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.600786 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:14:24.600988 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.598042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:34815 (local address 127.2.119.193:43291)
0619 08:14:24.598084 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.598086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.598102 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:14:24.598161 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.598162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.598171 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.598228 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:24.598419 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.598972 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600171 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.600591 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600664 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.600671 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.600879 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.600920 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:14:24.601048 4764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.597863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43291 (local address 127.2.119.195:34815)
0619 08:14:24.598027 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.598038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.598093 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.598243 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.598243 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.598244 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:24.598335 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.598337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.599070 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.599071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.600082 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.600084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.600693 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.600694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.600789 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.600793 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.600851 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.600917 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":13,"thread_start_us":85,"threads_started":1}
I20260619 08:14:24.601179 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f984a71b964a1fa87769e186ecff3d" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4" is_pre_election: true
I20260619 08:14:24.601390 4549 raft_consensus.cc:2454] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 0.
I20260619 08:14:24.601398 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f984a71b964a1fa87769e186ecff3d" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb" is_pre_election: true
I20260619 08:14:24.601522 4409 raft_consensus.cc:2454] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 0.
I20260619 08:14:24.601629 4621 leader_election.cc:302] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09dce63eab6048ca93802984739cf055, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:24.601787 4761 raft_consensus.cc:2781] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:24.601837 4761 raft_consensus.cc:482] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:24.601853 4761 raft_consensus.cc:3037] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.602609 4761 raft_consensus.cc:504] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.602748 4761 leader_election.cc:288] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 election: Requested vote from peers ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297), 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291)
I20260619 08:14:24.602885 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f984a71b964a1fa87769e186ecff3d" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4"
I20260619 08:14:24.602895 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6f984a71b964a1fa87769e186ecff3d" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb"
I20260619 08:14:24.602962 4549 raft_consensus.cc:3037] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.602962 4409 raft_consensus.cc:3037] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.603859 4409 raft_consensus.cc:2454] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 1.
I20260619 08:14:24.603859 4549 raft_consensus.cc:2454] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 1.
I20260619 08:14:24.604024 4621 leader_election.cc:302] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09dce63eab6048ca93802984739cf055, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:24.604135 4761 raft_consensus.cc:2781] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:24.604324 4761 raft_consensus.cc:686] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 1 LEADER]: Becoming Leader. State: Replica: 09dce63eab6048ca93802984739cf055, State: Running, Role: LEADER
I20260619 08:14:24.604436 4761 consensus_queue.cc:237] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } }
I20260619 08:14:24.605577 4251 catalog_manager.cc:5495] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09dce63eab6048ca93802984739cf055 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } health_report { overall_health: UNKNOWN } } }
W20260619 08:14:24.616935 4456 tablet.cc:2369] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 08:14:24 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856863, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:24.665238 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.658152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.2.119.254:42723)
0619 08:14:24.658257 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.658267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.658375 (+ 108us) server_negotiation.cc:407] Connection header received
0619 08:14:24.658489 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.658491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.658502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.658554 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:24.659057 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.659556 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.660297 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.660502 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.660583 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.662815 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:24.662816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:24.662817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:24.662825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:24.664032 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.664453 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.664454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.664454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.664511 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:24.664704 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:24.664705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:24.664705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:24.664819 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:24.664894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.665044 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.665128 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 08:14:24.665362 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.658044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42723 (local address 127.0.0.1:41726)
0619 08:14:24.658279 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.658291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.658391 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.658558 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.658559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.658908 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:24.658956 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.658958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.659635 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.659636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.660196 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.660198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.660591 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.660592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.660658 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.661226 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:24.661233 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:24.662716 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:24.664155 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.664162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.664165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.664363 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.664573 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:24.664574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:24.664575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:24.664637 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:24.664884 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:24.664892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:24.664939 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.665021 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.665205 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":78,"thread_start_us":126,"threads_started":1}
I20260619 08:14:24.667689 4251 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:41726:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:14:24.667956 4251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260619 08:14:24.668015 4736 tablet.cc:2369] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:24.669544 4389 tablet_service.cc:1467] Processing CreateTablet for tablet 92b67e0f092d4cb598c908119887922f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0dc6f0db7cd49989d43a13000a95101]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:24.669559 4669 tablet_service.cc:1467] Processing CreateTablet for tablet 92b67e0f092d4cb598c908119887922f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0dc6f0db7cd49989d43a13000a95101]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:24.669564 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 92b67e0f092d4cb598c908119887922f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c0dc6f0db7cd49989d43a13000a95101]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:24.669730 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b67e0f092d4cb598c908119887922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.669730 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b67e0f092d4cb598c908119887922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.669732 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b67e0f092d4cb598c908119887922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.670976 4757 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: Bootstrap starting.
I20260619 08:14:24.671245 4755 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: Bootstrap starting.
I20260619 08:14:24.671240 4756 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: Bootstrap starting.
I20260619 08:14:24.671434 4757 tablet_bootstrap.cc:654] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.671663 4755 tablet_bootstrap.cc:654] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.671938 4756 tablet_bootstrap.cc:654] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.672154 4757 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: No bootstrap required, opened a new log
I20260619 08:14:24.672195 4757 ts_tablet_manager.cc:1397] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:24.672363 4757 raft_consensus.cc:348] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.672454 4757 raft_consensus.cc:374] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.672480 4757 raft_consensus.cc:729] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.672530 4757 consensus_queue.cc:260] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.672639 4757 ts_tablet_manager.cc:1428] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.672652 4756 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: No bootstrap required, opened a new log
I20260619 08:14:24.672695 4756 ts_tablet_manager.cc:1397] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260619 08:14:24.672672 4762 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.672744 4762 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.672848 4756 raft_consensus.cc:348] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.672927 4756 raft_consensus.cc:374] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.672952 4756 raft_consensus.cc:729] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.672995 4756 consensus_queue.cc:260] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.673063 4755 tablet_bootstrap.cc:492] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: No bootstrap required, opened a new log
I20260619 08:14:24.673079 4756 ts_tablet_manager.cc:1428] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.673115 4755 ts_tablet_manager.cc:1397] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:24.673202 4763 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.673266 4755 raft_consensus.cc:348] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.673370 4755 raft_consensus.cc:374] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.673398 4755 raft_consensus.cc:729] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dce63eab6048ca93802984739cf055, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.673445 4755 consensus_queue.cc:260] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.673285 4763 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.673547 4755 ts_tablet_manager.cc:1428] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:24.673666 4761 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.673729 4761 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.867679 4761 raft_consensus.cc:482] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:24.867779 4761 raft_consensus.cc:504] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.867941 4761 leader_election.cc:288] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297)
I20260619 08:14:24.868149 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b67e0f092d4cb598c908119887922f" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4" is_pre_election: true
I20260619 08:14:24.868170 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b67e0f092d4cb598c908119887922f" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb" is_pre_election: true
I20260619 08:14:24.868260 4409 raft_consensus.cc:2454] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 0.
I20260619 08:14:24.868260 4549 raft_consensus.cc:2454] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 0.
I20260619 08:14:24.868445 4620 leader_election.cc:302] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, 09dce63eab6048ca93802984739cf055; no voters:
I20260619 08:14:24.868559 4761 raft_consensus.cc:2781] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:24.868605 4761 raft_consensus.cc:482] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:24.868620 4761 raft_consensus.cc:3037] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.869256 4761 raft_consensus.cc:504] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.869451 4761 leader_election.cc:288] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 election: Requested vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297)
I20260619 08:14:24.869570 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b67e0f092d4cb598c908119887922f" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb"
I20260619 08:14:24.869647 4409 raft_consensus.cc:3037] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.869609 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b67e0f092d4cb598c908119887922f" candidate_uuid: "09dce63eab6048ca93802984739cf055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4"
I20260619 08:14:24.869715 4549 raft_consensus.cc:3037] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:24.870280 4409 raft_consensus.cc:2454] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 1.
I20260619 08:14:24.870440 4549 raft_consensus.cc:2454] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09dce63eab6048ca93802984739cf055 in term 1.
I20260619 08:14:24.870448 4620 leader_election.cc:302] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, 09dce63eab6048ca93802984739cf055; no voters:
I20260619 08:14:24.870581 4761 raft_consensus.cc:2781] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:24.870667 4761 raft_consensus.cc:686] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [term 1 LEADER]: Becoming Leader. State: Replica: 09dce63eab6048ca93802984739cf055, State: Running, Role: LEADER
I20260619 08:14:24.870723 4761 consensus_queue.cc:237] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.871071 4765 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: TxnStatusTablet state changed. Reason: New leader 09dce63eab6048ca93802984739cf055. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.871167 4765 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:24.871387 4777 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:14:24.871503 4777 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:14:24.871443 4252 catalog_manager.cc:5495] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09dce63eab6048ca93802984739cf055 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } health_report { overall_health: HEALTHY } } }
I20260619 08:14:24.902719 4772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.899624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.0.0.1:47648)
0619 08:14:24.899710 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.899723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.899797 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.899986 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.899987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.899988 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:24.900037 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.900040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.900983 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.900984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.901641 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.901644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.902170 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.902171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.902230 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.902246 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:24.902564 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:24.902574 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.902617 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.902660 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:14:24.902750 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.899730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.2.119.195:37033)
0619 08:14:24.899797 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.899799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.899811 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:14:24.899878 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.899879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.899900 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.899974 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:24.900100 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.900881 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.901739 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.902073 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.902143 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.902351 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:24.902478 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:24.902562 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.902630 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.902651 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:24.904820 4549 raft_consensus.cc:1261] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Refusing update from remote peer 09dce63eab6048ca93802984739cf055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:24.904851 4409 raft_consensus.cc:1261] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Refusing update from remote peer 09dce63eab6048ca93802984739cf055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:24.905128 4765 consensus_queue.cc:1035] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:24.905236 4761 consensus_queue.cc:1035] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:24.906494 4763 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: TxnStatusTablet state changed. Reason: New leader 09dce63eab6048ca93802984739cf055. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.906605 4763 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.907543 4765 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.907626 4765 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:24.907817 4762 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: TxnStatusTablet state changed. Reason: New leader 09dce63eab6048ca93802984739cf055. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.907912 4762 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.908155 4763 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.908231 4763 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P ea33030155ed4a18bff9192342c9e6c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.908707 4780 mvcc.cc:205] Tried to move back new op lower bound from 7298485718647353344 to 7298485718510813184. Current Snapshot: MvccSnapshot[applied={T|T < 7298485718647353344 or (T in {7298485718647353344})}]
I20260619 08:14:24.908710 4781 mvcc.cc:205] Tried to move back new op lower bound from 7298485718647353344 to 7298485718510813184. Current Snapshot: MvccSnapshot[applied={T|T < 7298485718647353344}]
I20260619 08:14:24.909166 4765 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.909240 4765 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 09dce63eab6048ca93802984739cf055: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:24.909744 4762 tablet_replica.cc:439] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "09dce63eab6048ca93802984739cf055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } } }
I20260619 08:14:24.909839 4762 tablet_replica.cc:442] T 92b67e0f092d4cb598c908119887922f P 094b609bfe914d6ca04bad6771d876fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:24.910408 4784 mvcc.cc:205] Tried to move back new op lower bound from 7298485718647353344 to 7298485718510813184. Current Snapshot: MvccSnapshot[applied={T|T < 7298485718647353344}]
I20260619 08:14:24.913537 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.911002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.2.119.195:37033)
0619 08:14:24.911037 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.911039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.911045 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:14:24.911065 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.911065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.911074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.911118 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:24.911305 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.912043 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.912825 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.913121 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.913191 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.913330 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:24.913362 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:24.913415 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.913452 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.913473 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:14:24.918721 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.916449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:60749 (local address 127.2.119.195:37033)
0619 08:14:24.916485 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:24.916486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:24.916522 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:14:24.916579 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:24.916579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:24.916587 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:24.916647 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:24.916796 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.917234 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.918001 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.918186 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.918277 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.918458 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:24.918490 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:24.918552 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:24.918622 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 08:14:24.918643 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":21}
I20260619 08:14:24.918727 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:24.916393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.2.119.195:60749)
0619 08:14:24.916465 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:24.916476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:24.916534 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:24.916648 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:24.916649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:24.916650 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:24.916716 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.916718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.917333 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.917334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.917903 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:24.917905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:24.918282 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:24.918283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:24.918339 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:24.918354 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:24.918572 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:24.918574 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:24.918621 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:14:24.918662 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":4}
I20260619 08:14:24.920965 4409 raft_consensus.cc:1261] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Refusing update from remote peer 09dce63eab6048ca93802984739cf055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:24.920967 4549 raft_consensus.cc:1261] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Refusing update from remote peer 09dce63eab6048ca93802984739cf055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:24.921152 4765 consensus_queue.cc:1035] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:24.921231 4761 consensus_queue.cc:1035] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:24.935213 4252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:24.935333 4252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:24.936133 4252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Sending DeleteTablet for 3 replicas of tablet e6f984a71b964a1fa87769e186ecff3d
I20260619 08:14:24.936451 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f984a71b964a1fa87769e186ecff3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:24 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55840
I20260619 08:14:24.936482 4669 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f984a71b964a1fa87769e186ecff3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:24 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:47634
I20260619 08:14:24.936810 4252 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:14:24.936873 4389 tablet_service.cc:1514] Processing DeleteTablet for tablet e6f984a71b964a1fa87769e186ecff3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:24 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:46698
W20260619 08:14:24.936944 4252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:24.937166 4796 tablet_replica.cc:330] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: stopping tablet replica
I20260619 08:14:24.937565 4796 raft_consensus.cc:2229] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:24.937929 4797 tablet_replica.cc:330] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: stopping tablet replica
I20260619 08:14:24.938048 4797 raft_consensus.cc:2229] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:24.938110 4797 raft_consensus.cc:2258] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:24.938222 4389 tablet_service.cc:1467] Processing CreateTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e (DEFAULT_TABLE table=test-table [id=83dac11bab3d425fac6beb3dc15e5336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.938361 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062e7e514f2e4b73bee44008fe3fdc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.938676 4797 ts_tablet_manager.cc:1905] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:24.939826 4757 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Bootstrap starting.
I20260619 08:14:24.940218 4757 tablet_bootstrap.cc:654] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.940497 4797 ts_tablet_manager.cc:1918] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:24.940562 4797 log.cc:1196] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/e6f984a71b964a1fa87769e186ecff3d
I20260619 08:14:24.940827 4797 ts_tablet_manager.cc:1939] T e6f984a71b964a1fa87769e186ecff3d P 094b609bfe914d6ca04bad6771d876fb: Deleting consensus metadata
I20260619 08:14:24.941255 4236 catalog_manager.cc:4841] TS 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291): tablet e6f984a71b964a1fa87769e186ecff3d (table test-table [id=c65c024c36244d13804470c01e7e8379]) successfully deleted
I20260619 08:14:24.941455 4757 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: No bootstrap required, opened a new log
I20260619 08:14:24.941516 4757 ts_tablet_manager.cc:1397] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:24.941660 4757 raft_consensus.cc:348] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.941733 4757 raft_consensus.cc:374] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.941772 4757 raft_consensus.cc:729] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.941824 4757 consensus_queue.cc:260] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.941942 4757 ts_tablet_manager.cc:1428] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:24.938182 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e (DEFAULT_TABLE table=test-table [id=83dac11bab3d425fac6beb3dc15e5336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.942308 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062e7e514f2e4b73bee44008fe3fdc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.943317 4756 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Bootstrap starting.
I20260619 08:14:24.937273 4795 tablet_replica.cc:330] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: stopping tablet replica
I20260619 08:14:24.943717 4756 tablet_bootstrap.cc:654] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.938213 4669 tablet_service.cc:1467] Processing CreateTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e (DEFAULT_TABLE table=test-table [id=83dac11bab3d425fac6beb3dc15e5336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:24.943909 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 062e7e514f2e4b73bee44008fe3fdc4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:24.944399 4756 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: No bootstrap required, opened a new log
I20260619 08:14:24.944454 4756 ts_tablet_manager.cc:1397] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:24.944597 4756 raft_consensus.cc:348] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.944672 4756 raft_consensus.cc:374] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.944694 4756 raft_consensus.cc:729] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.944736 4756 consensus_queue.cc:260] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.944821 4756 ts_tablet_manager.cc:1428] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.945204 4755 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Bootstrap starting.
I20260619 08:14:24.945515 4755 tablet_bootstrap.cc:654] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:24.946133 4755 tablet_bootstrap.cc:492] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: No bootstrap required, opened a new log
I20260619 08:14:24.946197 4755 ts_tablet_manager.cc:1397] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:24.946338 4755 raft_consensus.cc:348] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.946411 4755 raft_consensus.cc:374] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:24.946434 4755 raft_consensus.cc:729] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dce63eab6048ca93802984739cf055, State: Initialized, Role: FOLLOWER
I20260619 08:14:24.946480 4755 consensus_queue.cc:260] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:24.946565 4755 ts_tablet_manager.cc:1428] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:24.946700 4795 raft_consensus.cc:2229] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:24.946976 4796 raft_consensus.cc:2258] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:24.947090 4795 raft_consensus.cc:2258] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:24.947472 4796 ts_tablet_manager.cc:1905] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:24.947510 4795 ts_tablet_manager.cc:1905] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:24.949388 4796 ts_tablet_manager.cc:1918] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:24.949467 4796 log.cc:1196] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/e6f984a71b964a1fa87769e186ecff3d
I20260619 08:14:24.949627 4795 ts_tablet_manager.cc:1918] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:24.949734 4796 ts_tablet_manager.cc:1939] T e6f984a71b964a1fa87769e186ecff3d P 09dce63eab6048ca93802984739cf055: Deleting consensus metadata
I20260619 08:14:24.949741 4795 log.cc:1196] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/e6f984a71b964a1fa87769e186ecff3d
I20260619 08:14:24.949990 4795 ts_tablet_manager.cc:1939] T e6f984a71b964a1fa87769e186ecff3d P ea33030155ed4a18bff9192342c9e6c4: Deleting consensus metadata
I20260619 08:14:24.950160 4236 catalog_manager.cc:4841] TS 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033): tablet e6f984a71b964a1fa87769e186ecff3d (table test-table [id=c65c024c36244d13804470c01e7e8379]) successfully deleted
I20260619 08:14:24.950620 4237 catalog_manager.cc:4841] TS ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297): tablet e6f984a71b964a1fa87769e186ecff3d (table test-table [id=c65c024c36244d13804470c01e7e8379]) successfully deleted
I20260619 08:14:25.079403 4762 raft_consensus.cc:482] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:25.079509 4762 raft_consensus.cc:504] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:25.079897 4762 leader_election.cc:288] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:25.083714 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.080248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:44567 (local address 127.2.119.195:37033)
0619 08:14:25.080296 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:25.080298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:25.080430 (+ 132us) server_negotiation.cc:407] Connection header received
0619 08:14:25.080464 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:25.080466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:25.080478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:25.080565 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:25.080766 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.081532 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.082993 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.083359 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.083446 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.083457 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:25.083616 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:14:25.083652 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:25.083734 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.080091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.2.119.193:44567)
0619 08:14:25.080351 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:25.080360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:25.080416 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:25.080577 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:25.080578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:25.080580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:25.080645 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.080646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.081615 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.081615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.082875 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.082878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.083447 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.083448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.083530 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.083534 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:25.083595 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:14:25.083636 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":55,"thread_start_us":60,"threads_started":1}
I20260619 08:14:25.083987 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062e7e514f2e4b73bee44008fe3fdc4e" candidate_uuid: "094b609bfe914d6ca04bad6771d876fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055" is_pre_election: true
I20260619 08:14:25.084093 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.080184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:42813 (local address 127.2.119.194:42297)
0619 08:14:25.080262 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:25.080265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:25.080331 (+ 66us) server_negotiation.cc:407] Connection header received
0619 08:14:25.080627 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:25.080628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:25.080637 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:25.080691 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:25.080917 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.081426 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.082982 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.083538 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.083615 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.083624 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:25.083975 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 08:14:25.084023 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:14:25.084117 4688 raft_consensus.cc:2454] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 094b609bfe914d6ca04bad6771d876fb in term 0.
I20260619 08:14:25.084328 4340 leader_election.cc:302] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, 09dce63eab6048ca93802984739cf055; no voters:
I20260619 08:14:25.084408 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.080085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:42297 (local address 127.2.119.193:42813)
0619 08:14:25.080251 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:25.080263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:25.080338 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:25.080714 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:25.080715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:25.080717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:25.080779 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.080780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.081511 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.081513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.082875 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.082878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.083662 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.083663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.083746 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.083750 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:25.084262 (+ 512us) client_negotiation.cc:240] Negotiation successful
0619 08:14:25.084306 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":57,"thread_start_us":95,"threads_started":1}
I20260619 08:14:25.084517 4762 raft_consensus.cc:2781] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:25.084584 4762 raft_consensus.cc:482] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:25.084607 4762 raft_consensus.cc:3037] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:25.084645 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062e7e514f2e4b73bee44008fe3fdc4e" candidate_uuid: "094b609bfe914d6ca04bad6771d876fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4" is_pre_election: true
I20260619 08:14:25.084738 4549 raft_consensus.cc:2454] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 094b609bfe914d6ca04bad6771d876fb in term 0.
I20260619 08:14:25.085456 4762 raft_consensus.cc:504] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:25.085592 4762 leader_election.cc:288] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [CANDIDATE]: Term 1 election: Requested vote from peers ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:25.085714 4549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062e7e514f2e4b73bee44008fe3fdc4e" candidate_uuid: "094b609bfe914d6ca04bad6771d876fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea33030155ed4a18bff9192342c9e6c4"
I20260619 08:14:25.085788 4549 raft_consensus.cc:3037] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:25.085772 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "062e7e514f2e4b73bee44008fe3fdc4e" candidate_uuid: "094b609bfe914d6ca04bad6771d876fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055"
I20260619 08:14:25.085837 4688 raft_consensus.cc:3037] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:25.086515 4688 raft_consensus.cc:2454] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 094b609bfe914d6ca04bad6771d876fb in term 1.
I20260619 08:14:25.086515 4549 raft_consensus.cc:2454] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 094b609bfe914d6ca04bad6771d876fb in term 1.
I20260619 08:14:25.086712 4341 leader_election.cc:302] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, 09dce63eab6048ca93802984739cf055, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:25.086815 4762 raft_consensus.cc:2781] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:25.086966 4762 raft_consensus.cc:686] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 1 LEADER]: Becoming Leader. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Running, Role: LEADER
I20260619 08:14:25.087038 4762 consensus_queue.cc:237] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:25.087781 4252 catalog_manager.cc:5495] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 094b609bfe914d6ca04bad6771d876fb (127.2.119.193). New cstate: current_term: 1 leader_uuid: "094b609bfe914d6ca04bad6771d876fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:25.130118 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.127680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.2.119.193:43291)
0619 08:14:25.127794 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:25.127795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:25.127803 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:25.127832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:25.127832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:25.127850 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:25.127918 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:25.128069 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.128661 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.129414 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.129598 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.129654 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.129771 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:25.129871 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:25.129923 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:25.129987 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:14:25.130003 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260619 08:14:25.135103 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.132682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:49551 (local address 127.2.119.195:37033)
0619 08:14:25.132721 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:25.132722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:25.132896 (+ 174us) server_negotiation.cc:407] Connection header received
0619 08:14:25.132922 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:25.132923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:25.132932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:25.132988 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:25.133087 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.133735 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.134506 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.134695 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.134753 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.134914 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:25.134940 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:25.134979 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:25.135029 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 08:14:25.135048 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:14:25.135120 4808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.132602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.2.119.193:49551)
0619 08:14:25.132773 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:25.132784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:25.132847 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:25.133003 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:25.133004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:25.133005 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:25.133043 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.133045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.133821 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.133821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.134423 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.134425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.134780 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.134781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.134828 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.134842 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:25.134993 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:25.134994 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:25.135028 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 08:14:25.135058 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260619 08:14:25.137216 4549 raft_consensus.cc:1261] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Refusing update from remote peer 094b609bfe914d6ca04bad6771d876fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:25.137235 4688 raft_consensus.cc:1261] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Refusing update from remote peer 094b609bfe914d6ca04bad6771d876fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:25.137527 4803 consensus_queue.cc:1035] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:25.137652 4762 consensus_queue.cc:1035] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:25.144901 4805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.142654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:37945 (local address 127.2.119.193:43291)
0619 08:14:25.142693 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:25.142695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:25.142747 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:14:25.142829 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:25.142830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:25.142840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:25.142893 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:25.142989 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.143436 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.144202 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.144472 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.144546 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.144687 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:25.144718 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:25.144764 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:25.144817 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 08:14:25.144836 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:14:25.144919 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:25.142611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43291 (local address 127.2.119.195:37945)
0619 08:14:25.142706 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:25.142717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:25.142761 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:25.142896 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:25.142896 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:25.142898 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:25.142935 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.142936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.143513 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.143514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.144135 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:25.144138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:25.144543 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:25.144544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:25.144598 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:25.144602 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:25.144763 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:25.144764 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:25.144813 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:14:25.144851 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
W20260619 08:14:25.145278 4784 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:14:26.668440 4250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:26.668572 4250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:26.669644 4250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Sending DeleteTablet for 3 replicas of tablet 062e7e514f2e4b73bee44008fe3fdc4e
I20260619 08:14:26.669992 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55840
I20260619 08:14:26.670006 4669 tablet_service.cc:1514] Processing DeleteTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:47634
I20260619 08:14:26.670027 4389 tablet_service.cc:1514] Processing DeleteTablet for tablet 062e7e514f2e4b73bee44008fe3fdc4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:46698
I20260619 08:14:26.670287 4835 tablet_replica.cc:330] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: stopping tablet replica
I20260619 08:14:26.670360 4835 raft_consensus.cc:2229] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:26.670408 4834 tablet_replica.cc:330] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: stopping tablet replica
I20260619 08:14:26.670491 4835 raft_consensus.cc:2258] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.670496 4834 raft_consensus.cc:2229] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:26.670536 4834 raft_consensus.cc:2258] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.670790 4834 ts_tablet_manager.cc:1905] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.670796 4835 ts_tablet_manager.cc:1905] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.672426 4833 tablet_replica.cc:330] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: stopping tablet replica
I20260619 08:14:26.672497 4833 raft_consensus.cc:2229] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:26.672595 4833 raft_consensus.cc:2258] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.672681 4835 ts_tablet_manager.cc:1918] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:14:26.672693 4834 ts_tablet_manager.cc:1918] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:14:26.672742 4835 log.cc:1196] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/062e7e514f2e4b73bee44008fe3fdc4e
I20260619 08:14:26.672753 4834 log.cc:1196] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/062e7e514f2e4b73bee44008fe3fdc4e
I20260619 08:14:26.672856 4833 ts_tablet_manager.cc:1905] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.673012 4834 ts_tablet_manager.cc:1939] T 062e7e514f2e4b73bee44008fe3fdc4e P ea33030155ed4a18bff9192342c9e6c4: Deleting consensus metadata
I20260619 08:14:26.673072 4835 ts_tablet_manager.cc:1939] T 062e7e514f2e4b73bee44008fe3fdc4e P 094b609bfe914d6ca04bad6771d876fb: Deleting consensus metadata
I20260619 08:14:26.673358 4237 catalog_manager.cc:4841] TS ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297): tablet 062e7e514f2e4b73bee44008fe3fdc4e (table test-table [id=83dac11bab3d425fac6beb3dc15e5336]) successfully deleted
I20260619 08:14:26.673605 4236 catalog_manager.cc:4841] TS 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291): tablet 062e7e514f2e4b73bee44008fe3fdc4e (table test-table [id=83dac11bab3d425fac6beb3dc15e5336]) successfully deleted
I20260619 08:14:26.674482 4833 ts_tablet_manager.cc:1918] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:14:26.674567 4833 log.cc:1196] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/062e7e514f2e4b73bee44008fe3fdc4e
I20260619 08:14:26.674800 4833 ts_tablet_manager.cc:1939] T 062e7e514f2e4b73bee44008fe3fdc4e P 09dce63eab6048ca93802984739cf055: Deleting consensus metadata
I20260619 08:14:26.675137 4236 catalog_manager.cc:4841] TS 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033): tablet 062e7e514f2e4b73bee44008fe3fdc4e (table test-table [id=83dac11bab3d425fac6beb3dc15e5336]) successfully deleted
Jun 19 08:14:26 dist-test-slave-d0b5 krb5kdc[4207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:14:26 dist-test-slave-d0b5 krb5kdc[4207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856866, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:26.700980 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.688314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.2.119.254:42723)
0619 08:14:26.688468 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.688470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.688480 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:26.688553 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.688554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.688564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.688642 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:26.689184 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.689798 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.690562 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.690770 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.690841 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.697970 (+ 7129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:26.697972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:26.697972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:26.697979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:26.699690 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:26.700116 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:26.700117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:26.700118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:26.700185 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:26.700395 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:26.700396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:26.700396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:26.700546 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:26.700658 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.700763 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.700853 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260619 08:14:26.702620 4250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:26.702783 4250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:26.704308 4389 tablet_service.cc:1467] Processing CreateTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 (DEFAULT_TABLE table=test-table [id=3dc04accd0b24e2285db0471d4c17f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.704321 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 (DEFAULT_TABLE table=test-table [id=3dc04accd0b24e2285db0471d4c17f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.704321 4669 tablet_service.cc:1467] Processing CreateTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 (DEFAULT_TABLE table=test-table [id=3dc04accd0b24e2285db0471d4c17f2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.704454 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b4f63961444fecb9dc0dddf6a432a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.704459 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b4f63961444fecb9dc0dddf6a432a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.704468 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68b4f63961444fecb9dc0dddf6a432a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.705932 4844 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Bootstrap starting.
I20260619 08:14:26.706065 4846 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Bootstrap starting.
I20260619 08:14:26.706386 4846 tablet_bootstrap.cc:654] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.706398 4844 tablet_bootstrap.cc:654] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.706717 4845 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Bootstrap starting.
I20260619 08:14:26.707065 4846 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: No bootstrap required, opened a new log
I20260619 08:14:26.707139 4845 tablet_bootstrap.cc:654] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.707188 4846 ts_tablet_manager.cc:1397] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:26.707232 4844 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: No bootstrap required, opened a new log
I20260619 08:14:26.707279 4844 ts_tablet_manager.cc:1397] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:26.707410 4844 raft_consensus.cc:348] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.707405 4846 raft_consensus.cc:348] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.707504 4846 raft_consensus.cc:374] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.707504 4844 raft_consensus.cc:374] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.707561 4846 raft_consensus.cc:729] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.707616 4844 raft_consensus.cc:729] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.707638 4846 consensus_queue.cc:260] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.707681 4845 tablet_bootstrap.cc:492] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: No bootstrap required, opened a new log
I20260619 08:14:26.707720 4845 ts_tablet_manager.cc:1397] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:26.707703 4844 consensus_queue.cc:260] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.707831 4846 ts_tablet_manager.cc:1428] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:26.707870 4844 ts_tablet_manager.cc:1428] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:26.707835 4845 raft_consensus.cc:348] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.707911 4845 raft_consensus.cc:374] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.707932 4845 raft_consensus.cc:729] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dce63eab6048ca93802984739cf055, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.707978 4845 consensus_queue.cc:260] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.708076 4845 ts_tablet_manager.cc:1428] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:26.804198 4850 raft_consensus.cc:482] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:26.804294 4850 raft_consensus.cc:504] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.804579 4850 leader_election.cc:288] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:26.807685 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.804820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43291 (local address 127.2.119.194:47707)
0619 08:14:26.804987 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:26.804999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:26.805055 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:26.805263 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:26.805264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:26.805267 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:26.805369 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.805371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.806031 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.806032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.806815 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.806817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.807390 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.807392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.807472 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.807476 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:26.807551 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:14:26.807594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260619 08:14:26.807674 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.804895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:47707 (local address 127.2.119.193:43291)
0619 08:14:26.805076 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.805078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.805086 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:26.805133 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.805134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.805147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.805254 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:26.805457 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.805949 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.806913 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.807308 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.807394 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.807402 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.807556 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.807602 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":119,"threads_started":1}
I20260619 08:14:26.807897 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b4f63961444fecb9dc0dddf6a432a7" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb" is_pre_election: true
I20260619 08:14:26.808012 4409 raft_consensus.cc:2454] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 0.
I20260619 08:14:26.808199 4480 leader_election.cc:302] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:26.808352 4850 raft_consensus.cc:2781] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:26.808411 4850 raft_consensus.cc:482] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:26.808442 4850 raft_consensus.cc:3037] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.808492 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.805237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:33819 (local address 127.2.119.195:37033)
0619 08:14:26.805407 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.805409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.805415 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:14:26.805434 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.805435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.805442 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.805514 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:26.805752 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.806213 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.807582 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.808146 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.808215 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.808222 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.808368 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.808413 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260619 08:14:26.808530 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.805039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.2.119.194:33819)
0619 08:14:26.805173 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:26.805206 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:26.805256 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:26.805532 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:26.805533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:26.805536 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:26.805617 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.805618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.806300 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.806301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.807461 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.807463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.808231 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.808232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.808303 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.808307 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:26.808362 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:14:26.808397 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260619 08:14:26.808717 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b4f63961444fecb9dc0dddf6a432a7" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055" is_pre_election: true
I20260619 08:14:26.808784 4688 raft_consensus.cc:2454] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 0.
I20260619 08:14:26.809341 4850 raft_consensus.cc:504] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.809460 4850 leader_election.cc:288] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 election: Requested vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:26.809645 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b4f63961444fecb9dc0dddf6a432a7" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb"
I20260619 08:14:26.809693 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68b4f63961444fecb9dc0dddf6a432a7" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055"
I20260619 08:14:26.809772 4688 raft_consensus.cc:3037] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.809764 4409 raft_consensus.cc:3037] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.810392 4688 raft_consensus.cc:2454] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 1.
I20260619 08:14:26.810457 4409 raft_consensus.cc:2454] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 1.
I20260619 08:14:26.810575 4480 leader_election.cc:302] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09dce63eab6048ca93802984739cf055, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:26.810714 4850 raft_consensus.cc:2781] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:26.810879 4850 raft_consensus.cc:686] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 1 LEADER]: Becoming Leader. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Running, Role: LEADER
I20260619 08:14:26.810990 4850 consensus_queue.cc:237] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.811731 4250 catalog_manager.cc:5495] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "ea33030155ed4a18bff9192342c9e6c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:26.827610 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.824991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.2.119.194:42297)
0619 08:14:26.825174 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.825176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.825183 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:26.825225 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.825226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.825243 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.825363 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:26.825507 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.826019 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.826784 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.827010 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.827087 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.827238 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:26.827359 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:26.827443 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.827484 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.827510 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260619 08:14:26.832520 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.830264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:51857 (local address 127.2.119.195:37033)
0619 08:14:26.830318 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.830320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.830329 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:26.830374 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.830375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.830386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.830459 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:26.830581 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.831052 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.831812 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.832010 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.832086 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.832269 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:26.832306 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:26.832362 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.832423 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.832445 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:14:26.832553 4862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.830097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37033 (local address 127.2.119.194:51857)
0619 08:14:26.830254 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:26.830264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:26.830315 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:26.830461 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:26.830462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:26.830463 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:26.830520 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.830521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.831137 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.831138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.831727 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.831728 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.832097 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.832098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.832160 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.832178 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:26.832378 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:26.832380 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:26.832421 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 08:14:26.832464 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260619 08:14:26.834985 4688 raft_consensus.cc:1261] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Refusing update from remote peer ea33030155ed4a18bff9192342c9e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:26.835021 4409 raft_consensus.cc:1261] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Refusing update from remote peer ea33030155ed4a18bff9192342c9e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:26.835223 4850 consensus_queue.cc:1035] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:26.835348 4855 consensus_queue.cc:1035] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:26.842953 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.840542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:59239 (local address 127.2.119.194:42297)
0619 08:14:26.840587 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:26.840589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:26.840693 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:14:26.840765 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:26.840766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:26.840777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:26.840837 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:26.840972 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.841556 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.842281 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.842485 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.842558 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.842709 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:26.842740 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:26.842810 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:26.842865 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:14:26.842885 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:14:26.842984 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:26.840490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:42297 (local address 127.2.119.195:59239)
0619 08:14:26.840624 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:26.840634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:26.840692 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:26.840842 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:26.840843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:26.840849 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:26.840891 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.840892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.841625 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.841626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.842185 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:26.842186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:26.842556 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:26.842557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:26.842614 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:26.842618 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:26.842819 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:26.842821 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:26.842864 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:14:26.842907 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260619 08:14:26.850088 4250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:26.850206 4250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:26.850814 4250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Sending DeleteTablet for 3 replicas of tablet 68b4f63961444fecb9dc0dddf6a432a7
I20260619 08:14:26.851076 4389 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:46698
I20260619 08:14:26.851094 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55840
I20260619 08:14:26.851197 4835 tablet_replica.cc:330] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: stopping tablet replica
I20260619 08:14:26.851199 4834 tablet_replica.cc:330] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: stopping tablet replica
I20260619 08:14:26.851255 4835 raft_consensus.cc:2229] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:26.851255 4834 raft_consensus.cc:2229] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:26.851291 4835 raft_consensus.cc:2258] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.851336 4834 raft_consensus.cc:2258] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.851678 4834 ts_tablet_manager.cc:1905] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.851687 4835 ts_tablet_manager.cc:1905] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.852978 4669 tablet_service.cc:1514] Processing DeleteTablet for tablet 68b4f63961444fecb9dc0dddf6a432a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:26 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:47634
I20260619 08:14:26.853113 4833 tablet_replica.cc:330] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: stopping tablet replica
I20260619 08:14:26.853204 4833 raft_consensus.cc:2229] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:26.853260 4833 raft_consensus.cc:2258] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:26.853269 4834 ts_tablet_manager.cc:1918] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:26.853288 4835 ts_tablet_manager.cc:1918] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:26.853363 4835 log.cc:1196] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/68b4f63961444fecb9dc0dddf6a432a7
I20260619 08:14:26.853371 4834 log.cc:1196] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/68b4f63961444fecb9dc0dddf6a432a7
I20260619 08:14:26.853595 4834 ts_tablet_manager.cc:1939] T 68b4f63961444fecb9dc0dddf6a432a7 P ea33030155ed4a18bff9192342c9e6c4: Deleting consensus metadata
I20260619 08:14:26.853601 4835 ts_tablet_manager.cc:1939] T 68b4f63961444fecb9dc0dddf6a432a7 P 094b609bfe914d6ca04bad6771d876fb: Deleting consensus metadata
I20260619 08:14:26.853828 4250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:14:26.853902 4237 catalog_manager.cc:4841] TS ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297): tablet 68b4f63961444fecb9dc0dddf6a432a7 (table test-table [id=3dc04accd0b24e2285db0471d4c17f2a]) successfully deleted
W20260619 08:14:26.854001 4250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:26.854260 4833 ts_tablet_manager.cc:1905] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:26.854811 4236 catalog_manager.cc:4841] TS 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291): tablet 68b4f63961444fecb9dc0dddf6a432a7 (table test-table [id=3dc04accd0b24e2285db0471d4c17f2a]) successfully deleted
I20260619 08:14:26.855315 4529 tablet_service.cc:1467] Processing CreateTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 (DEFAULT_TABLE table=test-table [id=cee503787a034c8b99e88bc1542b0a39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.855338 4669 tablet_service.cc:1467] Processing CreateTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 (DEFAULT_TABLE table=test-table [id=cee503787a034c8b99e88bc1542b0a39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.855425 4529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25682c7d5d0f48bf9e889a2b01f95756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.855439 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25682c7d5d0f48bf9e889a2b01f95756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.855871 4833 ts_tablet_manager.cc:1918] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:26.855918 4833 log.cc:1196] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/68b4f63961444fecb9dc0dddf6a432a7
I20260619 08:14:26.856133 4833 ts_tablet_manager.cc:1939] T 68b4f63961444fecb9dc0dddf6a432a7 P 09dce63eab6048ca93802984739cf055: Deleting consensus metadata
I20260619 08:14:26.856621 4236 catalog_manager.cc:4841] TS 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033): tablet 68b4f63961444fecb9dc0dddf6a432a7 (table test-table [id=3dc04accd0b24e2285db0471d4c17f2a]) successfully deleted
I20260619 08:14:26.856720 4845 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Bootstrap starting.
I20260619 08:14:26.856966 4845 tablet_bootstrap.cc:654] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.857187 4846 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Bootstrap starting.
I20260619 08:14:26.857510 4845 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: No bootstrap required, opened a new log
I20260619 08:14:26.857554 4845 ts_tablet_manager.cc:1397] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:26.857569 4846 tablet_bootstrap.cc:654] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.857661 4845 raft_consensus.cc:348] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.857725 4845 raft_consensus.cc:374] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.857746 4845 raft_consensus.cc:729] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dce63eab6048ca93802984739cf055, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.857787 4845 consensus_queue.cc:260] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.857858 4389 tablet_service.cc:1467] Processing CreateTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 (DEFAULT_TABLE table=test-table [id=cee503787a034c8b99e88bc1542b0a39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:26.857903 4845 ts_tablet_manager.cc:1428] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:26.857954 4389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25682c7d5d0f48bf9e889a2b01f95756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:26.858810 4844 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb: Bootstrap starting.
I20260619 08:14:26.859097 4844 tablet_bootstrap.cc:654] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:26.859148 4846 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: No bootstrap required, opened a new log
I20260619 08:14:26.859192 4846 ts_tablet_manager.cc:1397] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:14:26.859385 4846 raft_consensus.cc:348] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.859460 4846 raft_consensus.cc:374] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.859485 4846 raft_consensus.cc:729] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.859534 4846 consensus_queue.cc:260] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.859638 4846 ts_tablet_manager.cc:1428] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:26.859663 4844 tablet_bootstrap.cc:492] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb: No bootstrap required, opened a new log
I20260619 08:14:26.859709 4844 ts_tablet_manager.cc:1397] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:26.859899 4844 raft_consensus.cc:348] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.859966 4844 raft_consensus.cc:374] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:26.859987 4844 raft_consensus.cc:729] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b609bfe914d6ca04bad6771d876fb, State: Initialized, Role: FOLLOWER
I20260619 08:14:26.860045 4844 consensus_queue.cc:260] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.860142 4844 ts_tablet_manager.cc:1428] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:26.965029 4855 raft_consensus.cc:482] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:26.965135 4855 raft_consensus.cc:504] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.965288 4855 leader_election.cc:288] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:26.965562 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25682c7d5d0f48bf9e889a2b01f95756" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb" is_pre_election: true
I20260619 08:14:26.965565 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25682c7d5d0f48bf9e889a2b01f95756" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055" is_pre_election: true
I20260619 08:14:26.965670 4409 raft_consensus.cc:2454] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 0.
I20260619 08:14:26.965677 4688 raft_consensus.cc:2454] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 0.
I20260619 08:14:26.965850 4480 leader_election.cc:302] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09dce63eab6048ca93802984739cf055, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:26.965967 4855 raft_consensus.cc:2781] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:26.966012 4855 raft_consensus.cc:482] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:26.966070 4855 raft_consensus.cc:3037] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.966789 4855 raft_consensus.cc:504] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.966928 4855 leader_election.cc:288] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 election: Requested vote from peers 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291), 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033)
I20260619 08:14:26.967150 4409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25682c7d5d0f48bf9e889a2b01f95756" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094b609bfe914d6ca04bad6771d876fb"
I20260619 08:14:26.967206 4688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25682c7d5d0f48bf9e889a2b01f95756" candidate_uuid: "ea33030155ed4a18bff9192342c9e6c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09dce63eab6048ca93802984739cf055"
I20260619 08:14:26.967249 4409 raft_consensus.cc:3037] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.967281 4688 raft_consensus.cc:3037] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:26.968010 4688 raft_consensus.cc:2454] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 1.
I20260619 08:14:26.968010 4409 raft_consensus.cc:2454] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea33030155ed4a18bff9192342c9e6c4 in term 1.
I20260619 08:14:26.968214 4480 leader_election.cc:302] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094b609bfe914d6ca04bad6771d876fb, ea33030155ed4a18bff9192342c9e6c4; no voters:
I20260619 08:14:26.968361 4855 raft_consensus.cc:2781] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:26.968432 4855 raft_consensus.cc:686] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 1 LEADER]: Becoming Leader. State: Replica: ea33030155ed4a18bff9192342c9e6c4, State: Running, Role: LEADER
I20260619 08:14:26.968492 4855 consensus_queue.cc:237] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } }
I20260619 08:14:26.969172 4250 catalog_manager.cc:5495] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "ea33030155ed4a18bff9192342c9e6c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea33030155ed4a18bff9192342c9e6c4" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 42297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:26.982403 4688 raft_consensus.cc:1261] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Refusing update from remote peer ea33030155ed4a18bff9192342c9e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:26.982427 4409 raft_consensus.cc:1261] T 25682c7d5d0f48bf9e889a2b01f95756 P 094b609bfe914d6ca04bad6771d876fb [term 1 FOLLOWER]: Refusing update from remote peer ea33030155ed4a18bff9192342c9e6c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:26.982661 4855 consensus_queue.cc:1035] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09dce63eab6048ca93802984739cf055" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:26.982758 4865 consensus_queue.cc:1035] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "094b609bfe914d6ca04bad6771d876fb" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:14:26.986937 4817 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 08:14:28.504511 4252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:28.504874 4252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:28.505913 4252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c08d1393123243c695ff4eda38864165: Sending DeleteTablet for 3 replicas of tablet 25682c7d5d0f48bf9e889a2b01f95756
I20260619 08:14:28.506165 4529 tablet_service.cc:1514] Processing DeleteTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:28 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55840
I20260619 08:14:28.506218 4389 tablet_service.cc:1514] Processing DeleteTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:28 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:46698
I20260619 08:14:28.506245 4668 tablet_service.cc:1514] Processing DeleteTablet for tablet 25682c7d5d0f48bf9e889a2b01f95756 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:28 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:47634
I20260619 08:14:28.506367 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 4324
I20260619 08:14:28.506990 4897 tablet_replica.cc:330] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: stopping tablet replica
I20260619 08:14:28.507069 4895 tablet_replica.cc:330] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: stopping tablet replica
I20260619 08:14:28.507072 4897 raft_consensus.cc:2229] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:28.507140 4897 raft_consensus.cc:2258] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:28.507143 4895 raft_consensus.cc:2229] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:28.507205 4895 raft_consensus.cc:2258] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:28.507498 4897 ts_tablet_manager.cc:1905] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:28.507500 4895 ts_tablet_manager.cc:1905] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:28.509809 4897 ts_tablet_manager.cc:1918] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:14:28.510088 4897 log.cc:1196] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/25682c7d5d0f48bf9e889a2b01f95756
I20260619 08:14:28.510295 4895 ts_tablet_manager.cc:1918] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 08:14:28.510340 4897 ts_tablet_manager.cc:1939] T 25682c7d5d0f48bf9e889a2b01f95756 P 09dce63eab6048ca93802984739cf055: Deleting consensus metadata
I20260619 08:14:28.510370 4895 log.cc:1196] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/25682c7d5d0f48bf9e889a2b01f95756
I20260619 08:14:28.510653 4895 ts_tablet_manager.cc:1939] T 25682c7d5d0f48bf9e889a2b01f95756 P ea33030155ed4a18bff9192342c9e6c4: Deleting consensus metadata
I20260619 08:14:28.510735 4236 catalog_manager.cc:4841] TS 09dce63eab6048ca93802984739cf055 (127.2.119.195:37033): tablet 25682c7d5d0f48bf9e889a2b01f95756 (table test-table [id=cee503787a034c8b99e88bc1542b0a39]) successfully deleted
I20260619 08:14:28.511088 4237 catalog_manager.cc:4841] TS ea33030155ed4a18bff9192342c9e6c4 (127.2.119.194:42297): tablet 25682c7d5d0f48bf9e889a2b01f95756 (table test-table [id=cee503787a034c8b99e88bc1542b0a39]) successfully deleted
W20260619 08:14:28.514642 4236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:43291 (error 108)
I20260619 08:14:28.514925 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 4464
I20260619 08:14:28.517547 4899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:28.515107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43291 (local address 127.0.0.1:46714)
0619 08:14:28.517404 (+ 2297us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:28.517470 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2269,"thread_start_us":62,"threads_started":1}
W20260619 08:14:28.517671 4236 catalog_manager.cc:4584] TS 094b609bfe914d6ca04bad6771d876fb (127.2.119.193:43291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25682c7d5d0f48bf9e889a2b01f95756: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43291: connect: Connection refused (error 111)
I20260619 08:14:28.521422 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 4604
I20260619 08:14:28.527594 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 4220
2026-06-19T08:14:28Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6105 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:28 dist-test-slave-d0b5 krb5kdc[4902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:28 dist-test-slave-d0b5 krb5kdc[4902](info): set up 2 sockets
Jun 19 08:14:28 dist-test-slave-d0b5 krb5kdc[4902](info): commencing operation
krb5kdc: starting...
W20260619 08:14:29.586062 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:29 dist-test-slave-d0b5 krb5kdc[4902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:29.708627 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41891
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:29.789495 4915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:29.789754 4915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:29.789798 4915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:29.791143 4915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:29.791209 4915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:29.791236 4915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:29.791268 4915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:29.791296 4915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:29.792867 4915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:29.793169 4915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:29.793434 4915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:29.798672 4923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:29.798681 4921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:29.798743 4915 server_base.cc:1034] running on GCE node
W20260619 08:14:29.798679 4920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:29.799226 4915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:29.799486 4915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:29.800639 4915 hybrid_clock.cc:648] HybridClock initialized: now 1781856869800624 us; error 27 us; skew 500 ppm
Jun 19 08:14:29 dist-test-slave-d0b5 krb5kdc[4902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856869, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:29.803308 4915 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:29.803743 4915 webserver.cc:460] Webserver started at http://127.2.119.254:33617/ using document root <none> and password file <none>
I20260619 08:14:29.803989 4915 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:29.804035 4915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:29.804170 4915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:29.805254 4915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "3e760a2db93042c1b1c805def6aad34d"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "f51e125e1717596cf2c651f62e2b4aad"
server_key_iv: "4f14a15bb54f69cc0cc0c056cf9377f1"
server_key_version: "encryptionkey@0"
I20260619 08:14:29.805683 4915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "3e760a2db93042c1b1c805def6aad34d"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "f51e125e1717596cf2c651f62e2b4aad"
server_key_iv: "4f14a15bb54f69cc0cc0c056cf9377f1"
server_key_version: "encryptionkey@0"
I20260619 08:14:29.807045 4915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:29.807911 4930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:29.808086 4915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:29.808192 4915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "3e760a2db93042c1b1c805def6aad34d"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "f51e125e1717596cf2c651f62e2b4aad"
server_key_iv: "4f14a15bb54f69cc0cc0c056cf9377f1"
server_key_version: "encryptionkey@0"
I20260619 08:14:29.808264 4915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:29.826876 4915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:29.828835 4915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:29.829016 4915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:29.833020 4915 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41891
I20260619 08:14:29.833086 4982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41891 every 8 connection(s)
I20260619 08:14:29.833472 4915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:29.834393 4983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:29.836724 4983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d: Bootstrap starting.
I20260619 08:14:29.837356 4983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:29.837673 4983 log.cc:824] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:29.838358 4983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d: No bootstrap required, opened a new log
I20260619 08:14:29.840010 4983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } }
I20260619 08:14:29.840152 4983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:29.840176 4983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e760a2db93042c1b1c805def6aad34d, State: Initialized, Role: FOLLOWER
I20260619 08:14:29.840358 4983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } }
I20260619 08:14:29.840425 4983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:29.840446 4983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:29.840516 4983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:29.841044 4983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } }
I20260619 08:14:29.841154 4983 leader_election.cc:302] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e760a2db93042c1b1c805def6aad34d; no voters:
I20260619 08:14:29.841411 4983 leader_election.cc:288] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:29.841465 4986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:29.841662 4983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:29.841821 4986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [term 1 LEADER]: Becoming Leader. State: Replica: 3e760a2db93042c1b1c805def6aad34d, State: Running, Role: LEADER
I20260619 08:14:29.842001 4986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } }
I20260619 08:14:29.842398 4988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e760a2db93042c1b1c805def6aad34d. Latest consensus state: current_term: 1 leader_uuid: "3e760a2db93042c1b1c805def6aad34d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } } }
I20260619 08:14:29.842410 4987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e760a2db93042c1b1c805def6aad34d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e760a2db93042c1b1c805def6aad34d" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41891 } } }
I20260619 08:14:29.842497 4988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:29.842509 4987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:29.842672 4995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:29.843247 4995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:29.843633 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 4915
I20260619 08:14:29.843739 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:29.843909 2527 external_mini_cluster.cc:1428] Setting key df3438743d3d7346d8ec7bdc04016087
I20260619 08:14:29.844908 4995 catalog_manager.cc:1347] Generated new cluster ID: 6881ebc05c194ec1ab90f955805349a3
I20260619 08:14:29.844964 4995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:29 dist-test-slave-d0b5 krb5kdc[4902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:29.855856 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:29.844530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.2.119.254:41891)
0619 08:14:29.844703 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:29.844712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:29.844750 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:14:29.844871 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:29.844879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:29.844935 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:29.845074 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:29.845785 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:29.847319 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:29.848376 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:29.849119 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:29.849233 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:29.852018 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:29.852022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:29.852023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:29.852036 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:29.854109 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:29.854516 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:29.854519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:29.854519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:29.854632 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:29.854852 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:29.854852 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:29.854853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:29.855193 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:29.855349 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:29.855453 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 08:14:29.855598 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260619 08:14:29.863879 4995 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:29.864646 4995 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:29.874713 4995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e760a2db93042c1b1c805def6aad34d: Generated new TSK 0
I20260619 08:14:29.874954 4995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:29.905121 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:29.986995 5009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:29.987196 5009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:29.987221 5009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:29.988377 5009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:29.988420 5009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:29.988447 5009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:29.989950 5009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:29.990190 5009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:29.990439 5009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:29.995505 5014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:29.995517 5017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:29.995545 5015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:29.995713 5009 server_base.cc:1034] running on GCE node
I20260619 08:14:29.995954 5009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:29.996162 5009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:29.997368 5009 hybrid_clock.cc:648] HybridClock initialized: now 1781856869997361 us; error 35 us; skew 500 ppm
Jun 19 08:14:29 dist-test-slave-d0b5 krb5kdc[4902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856869, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:29.999954 5009 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:30.000384 5009 webserver.cc:460] Webserver started at http://127.2.119.193:42403/ using document root <none> and password file <none>
I20260619 08:14:30.000625 5009 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:30.000692 5009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:30.000845 5009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:30.002000 5009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "79310a5b0010418ba88677269b674dc5"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "bdfd5f42cbf69ece57c8043f6f5a4cf3"
server_key_iv: "f0a50ce95cd8a1cccfbc88badaa418f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.002350 5009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "79310a5b0010418ba88677269b674dc5"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "bdfd5f42cbf69ece57c8043f6f5a4cf3"
server_key_iv: "f0a50ce95cd8a1cccfbc88badaa418f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.003655 5009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:30.004454 5024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.004649 5009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:30.004748 5009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "79310a5b0010418ba88677269b674dc5"
format_stamp: "Formatted at 2026-06-19 08:14:29 on dist-test-slave-d0b5"
server_key: "bdfd5f42cbf69ece57c8043f6f5a4cf3"
server_key_iv: "f0a50ce95cd8a1cccfbc88badaa418f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.004828 5009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:30.014196 5009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:30.016564 5009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:30.016731 5009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:30.016970 5009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:30.017279 5009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:30.017367 5009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.017403 5009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:30.017431 5009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.023304 5009 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:45241
I20260619 08:14:30.023350 5137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:45241 every 8 connection(s)
I20260619 08:14:30.023684 5009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[4902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856869, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:30.030313 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5009
I20260619 08:14:30.030408 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:30.030591 2527 external_mini_cluster.cc:1428] Setting key 97d77568e1dcb4e47de22e15457066d9
I20260619 08:14:30.034596 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.024495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54107 (local address 127.2.119.254:41891)
0619 08:14:30.024547 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:30.024549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:30.024768 (+ 219us) server_negotiation.cc:407] Connection header received
0619 08:14:30.025346 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:30.025348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:30.025375 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:30.025439 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:30.026398 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.026930 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.028079 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.028303 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.028371 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.030937 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:30.030938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:30.030938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:30.030950 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:30.032873 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.033528 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.033529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.033529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.033590 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.033903 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.033904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.033904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.034018 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:30.034074 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:30.034446 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 08:14:30.034527 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:30.035001 5140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.024415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41891 (local address 127.2.119.193:54107)
0619 08:14:30.024589 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:30.024632 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:30.025199 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:30.025518 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:30.025522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:30.026084 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:30.026282 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.026287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.027031 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.027032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.027984 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.027987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.028388 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.028389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.028496 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.029034 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:30.029043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:30.030796 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:30.033036 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.033045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.033048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.033441 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.033667 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.033668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.033668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.033827 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.034087 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:30.034093 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:30.034286 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 08:14:30.034409 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 08:14:30.034599 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260619 08:14:30.035549 5138 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41891
I20260619 08:14:30.035667 5138 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:30.035900 5138 heartbeater.cc:511] Master 127.2.119.254:41891 requested a full tablet report, sending...
I20260619 08:14:30.036471 4947 ts_manager.cc:194] Registered new tserver with Master: 79310a5b0010418ba88677269b674dc5 (127.2.119.193:45241)
I20260619 08:14:30.037225 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:54107
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:30.056622 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:30.138036 5143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:30.138266 5143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:30.138306 5143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:30.139462 5143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:30.139531 5143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:30.139575 5143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:30.141093 5143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:30.141371 5143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:30.141602 5143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:30.146813 5149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:30.146839 5151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:30.146896 5143 server_base.cc:1034] running on GCE node
W20260619 08:14:30.146865 5148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:30.147192 5143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:30.147401 5143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:30.148538 5143 hybrid_clock.cc:648] HybridClock initialized: now 1781856870148521 us; error 38 us; skew 500 ppm
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[4902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856870, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:30.151127 5143 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:30.151608 5143 webserver.cc:460] Webserver started at http://127.2.119.194:40221/ using document root <none> and password file <none>
I20260619 08:14:30.151850 5143 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:30.151918 5143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:30.152061 5143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:30.153081 5143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "f17f83e492194a76a7f68aa149c4a26c"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "9923a69f19e541aa601953b9d06d5927"
server_key_iv: "8527941673904ff209e8444f70f0252e"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.153461 5143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "f17f83e492194a76a7f68aa149c4a26c"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "9923a69f19e541aa601953b9d06d5927"
server_key_iv: "8527941673904ff209e8444f70f0252e"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.154758 5143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:30.155503 5158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.155669 5143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:30.155756 5143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "f17f83e492194a76a7f68aa149c4a26c"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "9923a69f19e541aa601953b9d06d5927"
server_key_iv: "8527941673904ff209e8444f70f0252e"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.155835 5143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:30.181298 5143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:30.183380 5143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:30.183547 5143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:30.183787 5143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:30.184104 5143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:30.184166 5143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.184207 5143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:30.184235 5143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.190446 5143 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:41201
I20260619 08:14:30.190495 5271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:41201 every 8 connection(s)
I20260619 08:14:30.190793 5143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:30.191767 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5143
I20260619 08:14:30.191872 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:30.192058 2527 external_mini_cluster.cc:1428] Setting key b3098cb533cf6b804a337993fa47730d
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[4902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856870, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:14:30.201120 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.191499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:49959 (local address 127.2.119.254:41891)
0619 08:14:30.191580 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:30.191582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:30.191834 (+ 252us) server_negotiation.cc:407] Connection header received
0619 08:14:30.192520 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:30.192521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:30.192530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:30.192575 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:30.193937 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.194619 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.195720 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.196055 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.196133 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.198312 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:30.198314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:30.198314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:30.198328 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:30.199689 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.200113 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.200114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.200114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.200174 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.200421 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.200422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.200422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.200534 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:30.200614 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:30.200945 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 08:14:30.201034 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:14:30.201292 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.191456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41891 (local address 127.2.119.194:49959)
0619 08:14:30.191671 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:30.191722 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:30.192395 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:30.192656 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:30.192660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:30.193552 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:30.193832 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.193839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.194732 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.194733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.195634 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.195636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.196140 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.196141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.196304 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.196838 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:30.196846 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:30.198208 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:30.199773 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.199780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.199782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.200035 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.200242 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.200243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.200243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.200353 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.200624 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:30.200630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:30.200799 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 08:14:30.200912 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:14:30.201064 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260619 08:14:30.201781 5272 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41891
I20260619 08:14:30.201884 5272 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:30.202059 5272 heartbeater.cc:511] Master 127.2.119.254:41891 requested a full tablet report, sending...
I20260619 08:14:30.202447 4947 ts_manager.cc:194] Registered new tserver with Master: f17f83e492194a76a7f68aa149c4a26c (127.2.119.194:41201)
I20260619 08:14:30.202884 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:49959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:30.220647 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:41741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:30.300778 5277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:30.300985 5277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:30.301007 5277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:30.302198 5277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:30.302239 5277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:30.302265 5277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:30.303682 5277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:30.303922 5277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:30.304147 5277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:30.309134 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:30.309163 5282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:30.309145 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:30.309568 5277 server_base.cc:1034] running on GCE node
I20260619 08:14:30.309845 5277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:30.310091 5277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:30.311234 5277 hybrid_clock.cc:648] HybridClock initialized: now 1781856870311222 us; error 45 us; skew 500 ppm
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[4902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856870, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:30.313866 5277 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:30.314234 5277 webserver.cc:460] Webserver started at http://127.2.119.195:39699/ using document root <none> and password file <none>
I20260619 08:14:30.314433 5277 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:30.314473 5277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:30.314558 5277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:30.315583 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "65e627562bbf4ad28fd5225df666ae3a"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "fc31a81ba187b5ea5c383ffbe05c017e"
server_key_iv: "56776db35db2eb93fdcc02dcba92e7c6"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.315868 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "65e627562bbf4ad28fd5225df666ae3a"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "fc31a81ba187b5ea5c383ffbe05c017e"
server_key_iv: "56776db35db2eb93fdcc02dcba92e7c6"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.317077 5277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:30.317842 5292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.317992 5277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.318075 5277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "65e627562bbf4ad28fd5225df666ae3a"
format_stamp: "Formatted at 2026-06-19 08:14:30 on dist-test-slave-d0b5"
server_key: "fc31a81ba187b5ea5c383ffbe05c017e"
server_key_iv: "56776db35db2eb93fdcc02dcba92e7c6"
server_key_version: "encryptionkey@0"
I20260619 08:14:30.318125 5277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:30.339213 5277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:30.341540 5277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:30.341657 5277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:30.341848 5277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:30.342149 5277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:30.342181 5277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.342203 5277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:30.342249 5277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:30.348186 5277 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:38637
I20260619 08:14:30.348263 5405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:38637 every 8 connection(s)
I20260619 08:14:30.348577 5277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[4902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856870, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:30.355526 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5277
I20260619 08:14:30.355628 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:30.355854 2527 external_mini_cluster.cc:1428] Setting key d61b82318bad9fc0761215d1ca762b54
I20260619 08:14:30.357640 5006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.349255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:50051 (local address 127.2.119.254:41891)
0619 08:14:30.349350 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:30.349352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:30.349549 (+ 197us) server_negotiation.cc:407] Connection header received
0619 08:14:30.349994 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:30.349995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:30.350005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:30.350056 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:30.350903 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.351384 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.352338 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.352525 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.352571 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.354763 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:30.354765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:30.354765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:30.354773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:30.356341 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.356804 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.356805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.356805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.356863 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:30.357069 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:30.357070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:30.357070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:30.357169 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:30.357216 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:30.357517 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 08:14:30.357578 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 08:14:30.357793 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:30.349217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41891 (local address 127.2.119.195:50051)
0619 08:14:30.349432 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:30.349466 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:30.349915 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:30.350110 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:30.350113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:30.350643 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:30.350833 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.350837 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.351459 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.351459 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.352258 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:30.352259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:30.352591 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:30.352592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:30.352695 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:30.353196 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:30.353211 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:30.354639 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:30.356421 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.356428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.356430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.356731 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.356921 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:30.356921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:30.356922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:30.357014 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:30.357220 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:30.357224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:30.357403 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 08:14:30.357488 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:14:30.357603 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":145,"threads_started":1}
I20260619 08:14:30.358230 5406 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41891
I20260619 08:14:30.358307 5406 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:30.358552 5406 heartbeater.cc:511] Master 127.2.119.254:41891 requested a full tablet report, sending...
I20260619 08:14:30.358947 4947 ts_manager.cc:194] Registered new tserver with Master: 65e627562bbf4ad28fd5225df666ae3a (127.2.119.195:38637)
I20260619 08:14:30.359306 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:50051
I20260619 08:14:30.367388 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:14:30.377836 5006 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:30.376939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.2.119.254:41891)
0619 08:14:30.376999 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:30.377001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:30.377144 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:14:30.377184 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:30.377185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:30.377194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:30.377241 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:30.377786 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45232: BlockingRecv error: recv got EOF from 127.0.0.1:45232 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:14:30.378696 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5009
I20260619 08:14:30.383322 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5143
I20260619 08:14:30.387571 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5277
I20260619 08:14:30.391702 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 4915
2026-06-19T08:14:30Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1862 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[5418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[5418](info): set up 2 sockets
Jun 19 08:14:30 dist-test-slave-d0b5 krb5kdc[5418](info): commencing operation
krb5kdc: starting...
W20260619 08:14:31.434144 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:31.553139 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40399
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:40399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:31.633876 5431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:31.634140 5431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:31.634177 5431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:31.635370 5431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:31.635447 5431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:31.635470 5431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:31.635492 5431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:31.635522 5431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:31.637015 5431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:40399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:31.637318 5431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:31.637554 5431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:14:31.642683 5431 server_base.cc:1034] running on GCE node
W20260619 08:14:31.642699 5437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:31.642807 5439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:31.642686 5436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:31.643196 5431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:31.643510 5431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:31.644687 5431 hybrid_clock.cc:648] HybridClock initialized: now 1781856871644688 us; error 43 us; skew 500 ppm
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856871, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:31.647352 5431 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:31.647775 5431 webserver.cc:460] Webserver started at http://127.2.119.254:42137/ using document root <none> and password file <none>
I20260619 08:14:31.647994 5431 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:31.648061 5431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:31.648208 5431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:31.649273 5431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "9f334907bf6640cbb3cf1af2f9eb3a97"
format_stamp: "Formatted at 2026-06-19 08:14:31 on dist-test-slave-d0b5"
server_key: "7e0a21fe641410444104560a0121c994"
server_key_iv: "ed0b2025186abd0496c9f163ff49fe33"
server_key_version: "encryptionkey@0"
I20260619 08:14:31.649690 5431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "9f334907bf6640cbb3cf1af2f9eb3a97"
format_stamp: "Formatted at 2026-06-19 08:14:31 on dist-test-slave-d0b5"
server_key: "7e0a21fe641410444104560a0121c994"
server_key_iv: "ed0b2025186abd0496c9f163ff49fe33"
server_key_version: "encryptionkey@0"
I20260619 08:14:31.651046 5431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:31.651795 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:31.652051 5431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:31.652169 5431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "9f334907bf6640cbb3cf1af2f9eb3a97"
format_stamp: "Formatted at 2026-06-19 08:14:31 on dist-test-slave-d0b5"
server_key: "7e0a21fe641410444104560a0121c994"
server_key_iv: "ed0b2025186abd0496c9f163ff49fe33"
server_key_version: "encryptionkey@0"
I20260619 08:14:31.652268 5431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:31.662184 5431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:31.664671 5431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:31.664846 5431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:31.669126 5431 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:40399
I20260619 08:14:31.669155 5498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:40399 every 8 connection(s)
I20260619 08:14:31.669591 5431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:31.670449 5499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:31.672746 5499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97: Bootstrap starting.
I20260619 08:14:31.673432 5499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:31.673733 5499 log.cc:824] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:31.674394 5499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97: No bootstrap required, opened a new log
I20260619 08:14:31.675907 5499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } }
I20260619 08:14:31.676041 5499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:31.676064 5499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f334907bf6640cbb3cf1af2f9eb3a97, State: Initialized, Role: FOLLOWER
I20260619 08:14:31.676198 5499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } }
I20260619 08:14:31.676272 5499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:31.676297 5499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:31.676331 5499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:31.677107 5499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } }
I20260619 08:14:31.677189 5499 leader_election.cc:302] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f334907bf6640cbb3cf1af2f9eb3a97; no voters:
I20260619 08:14:31.677373 5499 leader_election.cc:288] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:31.677377 5502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:31.677558 5502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [term 1 LEADER]: Becoming Leader. State: Replica: 9f334907bf6640cbb3cf1af2f9eb3a97, State: Running, Role: LEADER
I20260619 08:14:31.677664 5499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:31.677711 5502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } }
I20260619 08:14:31.677904 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5431
I20260619 08:14:31.677995 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:31.678160 5504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f334907bf6640cbb3cf1af2f9eb3a97. Latest consensus state: current_term: 1 leader_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } } }
I20260619 08:14:31.678202 2527 external_mini_cluster.cc:1428] Setting key 54200bd44e3e3a6e6b2e7c202b0be3be
I20260619 08:14:31.678237 5504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:31.678386 5503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f334907bf6640cbb3cf1af2f9eb3a97" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40399 } } }
I20260619 08:14:31.678437 5503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:31.678911 5510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:31.679392 5510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:31.681075 5510 catalog_manager.cc:1347] Generated new cluster ID: 6bfae6e0fda54265bac6fc9fa230872c
I20260619 08:14:31.681139 5510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:31.686872 5510 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:31.687528 5510 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:31.688763 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:31.678869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.2.119.254:40399)
0619 08:14:31.679189 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:31.679199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:31.679235 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:14:31.679365 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:31.679373 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:31.679437 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:31.679580 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:31.680332 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:31.681670 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:31.682398 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:31.682839 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:31.682976 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:31.685292 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:31.685317 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:31.685318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:31.685330 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:31.687263 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:31.687630 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:31.687633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:31.687633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:31.687730 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:31.687944 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:31.687945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:31.687945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:31.688164 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:31.688346 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:31.688411 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 08:14:31.688576 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":59,"threads_started":1}
I20260619 08:14:31.689599 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:14:31.692271 5510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f334907bf6640cbb3cf1af2f9eb3a97: Generated new TSK 0
I20260619 08:14:31.692433 5510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:14:31.708653 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:31.699969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34606 (local address 127.2.119.254:40399)
0619 08:14:31.700037 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:31.700040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:31.700169 (+ 129us) server_negotiation.cc:407] Connection header received
0619 08:14:31.700197 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:31.700198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:31.700210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:31.700299 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:31.700683 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:31.701519 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:31.702572 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:31.702906 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:31.702978 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:31.708101 (+ 5123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:31.708102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:31.708103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:31.708353 (+ 250us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:31.708373 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:31.708380 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:31.708443 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:31.708536 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:14:31.708566 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:14:31.711261 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5431
2026-06-19T08:14:31Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1317 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5532](info): set up 2 sockets
Jun 19 08:14:31 dist-test-slave-d0b5 krb5kdc[5532](info): commencing operation
krb5kdc: starting...
W20260619 08:14:32.754357 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:32 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:32.880115 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42199
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:42199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:32.966409 5545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:32.966624 5545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:32.966646 5545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:32.967819 5545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:32.967870 5545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:32.967885 5545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:32.967897 5545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:32.967909 5545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:32.969484 5545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:42199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:32.969728 5545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:32.969976 5545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:32.975071 5550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:32.975046 5553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:32.975167 5551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:32.975359 5545 server_base.cc:1034] running on GCE node
I20260619 08:14:32.975652 5545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:32.975948 5545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:32.977092 5545 hybrid_clock.cc:648] HybridClock initialized: now 1781856872977064 us; error 51 us; skew 500 ppm
Jun 19 08:14:32 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856872, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:32.979621 5545 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:32.980111 5545 webserver.cc:460] Webserver started at http://127.2.119.254:35289/ using document root <none> and password file <none>
I20260619 08:14:32.980353 5545 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:32.980423 5545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:32.980571 5545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:32.981777 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "9bea58eef47d4f698cefc653b1dbcd4b"
format_stamp: "Formatted at 2026-06-19 08:14:32 on dist-test-slave-d0b5"
server_key: "834006fc08d621a128979b151ca041f5"
server_key_iv: "6d24a74497386da62296bd77566f37fa"
server_key_version: "encryptionkey@0"
I20260619 08:14:32.982153 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "9bea58eef47d4f698cefc653b1dbcd4b"
format_stamp: "Formatted at 2026-06-19 08:14:32 on dist-test-slave-d0b5"
server_key: "834006fc08d621a128979b151ca041f5"
server_key_iv: "6d24a74497386da62296bd77566f37fa"
server_key_version: "encryptionkey@0"
I20260619 08:14:32.983492 5545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:32.984262 5560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:32.984444 5545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:32.984547 5545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "9bea58eef47d4f698cefc653b1dbcd4b"
format_stamp: "Formatted at 2026-06-19 08:14:32 on dist-test-slave-d0b5"
server_key: "834006fc08d621a128979b151ca041f5"
server_key_iv: "6d24a74497386da62296bd77566f37fa"
server_key_version: "encryptionkey@0"
I20260619 08:14:32.984622 5545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:32.996166 5545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:32.998279 5545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:32.998451 5545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:33.002557 5545 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:42199
I20260619 08:14:33.002606 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:42199 every 8 connection(s)
I20260619 08:14:33.002976 5545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:33.003938 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:33.005018 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5545
I20260619 08:14:33.005133 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:33.005389 2527 external_mini_cluster.cc:1428] Setting key a96a2cd622fc0b8b02bdb13f368a6bdf
I20260619 08:14:33.006876 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b: Bootstrap starting.
I20260619 08:14:33.007645 5613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:33.007969 5613 log.cc:824] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:33.008724 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b: No bootstrap required, opened a new log
I20260619 08:14:33.011154 5613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } }
I20260619 08:14:33.011353 5613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:33.011396 5613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bea58eef47d4f698cefc653b1dbcd4b, State: Initialized, Role: FOLLOWER
I20260619 08:14:33.011523 5613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } }
I20260619 08:14:33.011626 5613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:33.011665 5613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:33.011711 5613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:33.012496 5613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } }
I20260619 08:14:33.012596 5613 leader_election.cc:302] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bea58eef47d4f698cefc653b1dbcd4b; no voters:
I20260619 08:14:33.012799 5613 leader_election.cc:288] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:33.012830 5618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:33.012948 5618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [term 1 LEADER]: Becoming Leader. State: Replica: 9bea58eef47d4f698cefc653b1dbcd4b, State: Running, Role: LEADER
I20260619 08:14:33.013065 5613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:33.013083 5618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } }
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:33.013442 5618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bea58eef47d4f698cefc653b1dbcd4b. Latest consensus state: current_term: 1 leader_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } } }
I20260619 08:14:33.013527 5618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:33.013463 5619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bea58eef47d4f698cefc653b1dbcd4b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42199 } } }
I20260619 08:14:33.013706 5619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:33.014397 5624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:33.015179 5624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:33.017032 5624 catalog_manager.cc:1347] Generated new cluster ID: 31ced7dd369644729135cdf15eefb415
I20260619 08:14:33.017115 5624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:33.018396 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.005935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.2.119.254:42199)
0619 08:14:33.006190 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:33.006203 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:33.006233 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:14:33.006657 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:33.006663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:33.006709 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:33.006824 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:33.007741 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.009067 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.010177 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.010635 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.010726 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.013760 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:33.013764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:33.013765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:33.013778 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:33.016019 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.016418 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.016421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.016422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.016553 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.016937 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.016938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.016938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.017249 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:33.017527 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:33.018014 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 08:14:33.018163 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":108,"threads_started":1}
I20260619 08:14:33.043308 5624 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:33.043958 5624 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:33.051086 5624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bea58eef47d4f698cefc653b1dbcd4b: Generated new TSK 0
I20260619 08:14:33.051342 5624 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:33.081645 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:33.164472 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:33.164728 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:33.164772 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:33.166018 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:33.166098 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:33.166144 5639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:33.167658 5639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:33.167943 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:33.168181 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:33.173203 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:33.173269 5645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:33.173445 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:33.173573 5639 server_base.cc:1034] running on GCE node
I20260619 08:14:33.173795 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:33.174033 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:33.175225 5639 hybrid_clock.cc:648] HybridClock initialized: now 1781856873175217 us; error 37 us; skew 500 ppm
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:33.177804 5639 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:33.178299 5639 webserver.cc:460] Webserver started at http://127.2.119.193:38745/ using document root <none> and password file <none>
I20260619 08:14:33.178535 5639 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:33.178604 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:33.178740 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:33.179777 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "9789a86221444040936f412856d4d0db"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "ac23fdcf1287fd5d9cb3b008a4e98649"
server_key_iv: "28b0bc5d80f0ce0238b0e189381d665b"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.180148 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "9789a86221444040936f412856d4d0db"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "ac23fdcf1287fd5d9cb3b008a4e98649"
server_key_iv: "28b0bc5d80f0ce0238b0e189381d665b"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.181619 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:33.182354 5654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.182533 5639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:33.182648 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "9789a86221444040936f412856d4d0db"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "ac23fdcf1287fd5d9cb3b008a4e98649"
server_key_iv: "28b0bc5d80f0ce0238b0e189381d665b"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.182723 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:33.201081 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:33.202996 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:33.203164 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:33.203399 5639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:33.203722 5639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:33.203780 5639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.203816 5639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:33.203845 5639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.209905 5639 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:46549
I20260619 08:14:33.209947 5767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:46549 every 8 connection(s)
I20260619 08:14:33.210271 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:33.217056 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5639
I20260619 08:14:33.217141 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:33.217408 2527 external_mini_cluster.cc:1428] Setting key 8609d7e538add777b6999a228ec3ac63
I20260619 08:14:33.220530 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.211034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42199 (local address 127.2.119.193:50585)
0619 08:14:33.211192 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:33.211234 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:33.211807 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:33.212041 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:33.212045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:33.212628 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:33.212823 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.212827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.213781 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.213782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.214699 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.214701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.215147 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.215148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.215252 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.215737 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:33.215745 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:33.217206 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:33.219092 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.219101 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.219103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.219417 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.219637 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.219637 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.219638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.219737 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.220035 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:33.220040 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:33.220158 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 08:14:33.220246 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:14:33.220369 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260619 08:14:33.220532 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.211089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:50585 (local address 127.2.119.254:42199)
0619 08:14:33.211132 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:33.211135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:33.211325 (+ 190us) server_negotiation.cc:407] Connection header received
0619 08:14:33.211896 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:33.211897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:33.211922 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:33.211984 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:33.212917 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.213700 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.214793 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.215069 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.215127 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.217352 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:33.217358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:33.217358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:33.217366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:33.219006 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.219494 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.219495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.219495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.219566 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.219799 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.219800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.219800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.219944 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:33.220016 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:33.220294 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 08:14:33.220383 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:33.221096 5768 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42199
I20260619 08:14:33.221199 5768 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:33.221432 5768 heartbeater.cc:511] Master 127.2.119.254:42199 requested a full tablet report, sending...
I20260619 08:14:33.222018 5577 ts_manager.cc:194] Registered new tserver with Master: 9789a86221444040936f412856d4d0db (127.2.119.193:46549)
I20260619 08:14:33.222630 5577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:50585
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:33.245381 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:33.326494 5773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:33.326683 5773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:33.326706 5773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:33.327783 5773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:33.327818 5773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:33.327845 5773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:33.329344 5773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:33.329571 5773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:33.329805 5773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:33.334870 5778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:33.334864 5781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:33.334903 5779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:33.335184 5773 server_base.cc:1034] running on GCE node
I20260619 08:14:33.335425 5773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:33.335681 5773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:33.336841 5773 hybrid_clock.cc:648] HybridClock initialized: now 1781856873336838 us; error 74 us; skew 500 ppm
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:33.339393 5773 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:33.339893 5773 webserver.cc:460] Webserver started at http://127.2.119.194:46289/ using document root <none> and password file <none>
I20260619 08:14:33.340162 5773 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:33.340210 5773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:33.340338 5773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:33.341463 5773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "7beca83a3259401998272b50ca378ba5"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "82fcae8147cd04495a1346f77ba09a8f"
server_key_iv: "c7ba3319c61d60b6a380f16803b4a409"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.341817 5773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "7beca83a3259401998272b50ca378ba5"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "82fcae8147cd04495a1346f77ba09a8f"
server_key_iv: "c7ba3319c61d60b6a380f16803b4a409"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.343248 5773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:33.344168 5788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.344527 5773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:33.344602 5773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "7beca83a3259401998272b50ca378ba5"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "82fcae8147cd04495a1346f77ba09a8f"
server_key_iv: "c7ba3319c61d60b6a380f16803b4a409"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.344696 5773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:33.356477 5773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:33.358507 5773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:33.358618 5773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:33.358868 5773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:33.359196 5773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:33.359253 5773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.359283 5773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:33.359314 5773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.365017 5773 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:32965
I20260619 08:14:33.365069 5901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:32965 every 8 connection(s)
I20260619 08:14:33.365396 5773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:33.370191 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5773
I20260619 08:14:33.370276 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:33.370481 2527 external_mini_cluster.cc:1428] Setting key a8d684ab6de72e6370396cdd518ab0a5
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:33.375347 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.366079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:53019 (local address 127.2.119.254:42199)
0619 08:14:33.366152 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:33.366154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:33.366307 (+ 153us) server_negotiation.cc:407] Connection header received
0619 08:14:33.366845 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:33.366846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:33.366859 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:33.366896 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:33.367843 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.368347 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.369406 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.369615 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.369692 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.372104 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:33.372105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:33.372106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:33.372113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:33.373844 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.374300 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.374301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.374302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.374362 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.374620 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.374620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.374621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.374746 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:33.374824 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:33.375174 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 08:14:33.375256 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:14:33.375576 5904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.366040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42199 (local address 127.2.119.194:53019)
0619 08:14:33.366186 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:33.366227 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:33.366737 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:33.366979 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:33.366983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:33.367543 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:33.367731 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.367736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.368459 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.368460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.369273 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.369275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.369696 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.369697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.369827 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.370426 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:33.370435 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:33.372007 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:33.373973 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.373979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.373981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.374197 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.374431 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.374432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.374432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.374541 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.374815 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:33.374821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:33.375024 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 08:14:33.375139 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 08:14:33.375298 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260619 08:14:33.376044 5902 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42199
I20260619 08:14:33.376134 5902 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:33.376331 5902 heartbeater.cc:511] Master 127.2.119.254:42199 requested a full tablet report, sending...
I20260619 08:14:33.376742 5577 ts_manager.cc:194] Registered new tserver with Master: 7beca83a3259401998272b50ca378ba5 (127.2.119.194:32965)
I20260619 08:14:33.377341 5577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:53019
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:33.396996 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:33.476797 5907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:33.477013 5907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:33.477036 5907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:33.478259 5907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:33.478302 5907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:33.478328 5907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:33.479744 5907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:33.479974 5907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:33.480216 5907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:33.485337 5915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:33.485334 5913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:33.485508 5907 server_base.cc:1034] running on GCE node
W20260619 08:14:33.485570 5912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:33.485862 5907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:33.486083 5907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:33.487272 5907 hybrid_clock.cc:648] HybridClock initialized: now 1781856873487264 us; error 36 us; skew 500 ppm
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:33.489936 5907 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:33.490329 5907 webserver.cc:460] Webserver started at http://127.2.119.195:46487/ using document root <none> and password file <none>
I20260619 08:14:33.490561 5907 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:33.490605 5907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:33.490733 5907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:33.491834 5907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "f1d411867af44375bfc2d76392bd1389"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "21549911127bf8d7ef04fdf3b94e86cd"
server_key_iv: "0b908c0373dcd3fcaca15bd6ca6ffbfc"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.492177 5907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "f1d411867af44375bfc2d76392bd1389"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "21549911127bf8d7ef04fdf3b94e86cd"
server_key_iv: "0b908c0373dcd3fcaca15bd6ca6ffbfc"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.493664 5907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:33.494500 5922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.494889 5907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:33.494992 5907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "f1d411867af44375bfc2d76392bd1389"
format_stamp: "Formatted at 2026-06-19 08:14:33 on dist-test-slave-d0b5"
server_key: "21549911127bf8d7ef04fdf3b94e86cd"
server_key_iv: "0b908c0373dcd3fcaca15bd6ca6ffbfc"
server_key_version: "encryptionkey@0"
I20260619 08:14:33.495069 5907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:33.512353 5907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:33.515053 5907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:33.515221 5907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:33.515475 5907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:33.515789 5907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:33.515822 5907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.515873 5907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:33.515899 5907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:33.522006 5907 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:42271
I20260619 08:14:33.522078 6035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:42271 every 8 connection(s)
I20260619 08:14:33.522387 5907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:33.531893 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.523274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:60269 (local address 127.2.119.254:42199)
0619 08:14:33.523382 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:33.523385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:33.523552 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:14:33.524105 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:33.524106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:33.524118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:33.524176 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:33.525127 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.525720 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.526732 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.526937 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.527022 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.529183 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:33.529185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:33.529185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:33.529192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:33.530610 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.530979 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.530980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.530980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.531035 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.531312 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.531313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.531314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.531435 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:33.531496 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:33.531731 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:14:33.531795 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:33.532045 6038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.523201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42199 (local address 127.2.119.195:60269)
0619 08:14:33.523418 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:33.523463 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:33.523995 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:33.524255 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:33.524258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:33.524819 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:33.525026 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.525031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.525827 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.525828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.526630 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:33.526632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.527031 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.527032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:33.527161 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.527682 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:33.527690 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:33.529071 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:33.530687 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.530692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.530694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.530905 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.531096 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:33.531097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:33.531097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:33.531190 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:33.531489 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:33.531493 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:33.531608 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:14:33.531694 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:14:33.531811 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":89,"threads_started":1}
I20260619 08:14:33.532215 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 5907
I20260619 08:14:33.532289 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:33.532503 6036 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42199
I20260619 08:14:33.532529 2527 external_mini_cluster.cc:1428] Setting key 0b7eb33b3851d2fdc52ed7d99364ace7
I20260619 08:14:33.532603 6036 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:33.532775 6036 heartbeater.cc:511] Master 127.2.119.254:42199 requested a full tablet report, sending...
I20260619 08:14:33.533211 5577 ts_manager.cc:194] Registered new tserver with Master: f1d411867af44375bfc2d76392bd1389 (127.2.119.195:42271)
I20260619 08:14:33.533548 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:33.533665 5577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:60269
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[5532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856873, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:33.556656 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:33.549290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.2.119.254:42199)
0619 08:14:33.549377 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:33.549379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:33.549485 (+ 106us) server_negotiation.cc:407] Connection header received
0619 08:14:33.549573 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:33.549574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:33.549591 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:33.549656 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:33.550275 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.550778 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.551571 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:33.551770 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:33.551854 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:33.554262 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:33.554263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:33.554264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:33.554270 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:33.555665 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.556008 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.556009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.556009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.556058 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:33.556224 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:33.556225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:33.556225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:33.556361 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:33.556434 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:33.556528 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:14:33.556591 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 08:14:33.556931 5577 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56548
I20260619 08:14:33.557523 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5639
I20260619 08:14:33.562060 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5773
I20260619 08:14:33.566680 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5907
I20260619 08:14:33.571167 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 5545
2026-06-19T08:14:33Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1862 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[6048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[6048](info): set up 2 sockets
Jun 19 08:14:33 dist-test-slave-d0b5 krb5kdc[6048](info): commencing operation
krb5kdc: starting...
W20260619 08:14:34.606343 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:34 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:34.725226 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43555
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:43555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:34.807415 6061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:34.807610 6061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:34.807631 6061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:34.808800 6061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:34.808843 6061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:34.808858 6061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:34.808871 6061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:34.808912 6061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:34.810415 6061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:43555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:34.810662 6061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:34.810901 6061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:34.815680 6067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:34.815732 6066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:34.815693 6069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:34.815980 6061 server_base.cc:1034] running on GCE node
I20260619 08:14:34.816238 6061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:34.816536 6061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:34.817727 6061 hybrid_clock.cc:648] HybridClock initialized: now 1781856874817700 us; error 66 us; skew 500 ppm
Jun 19 08:14:34 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856874, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:34.820246 6061 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:34.820722 6061 webserver.cc:460] Webserver started at http://127.2.119.254:42517/ using document root <none> and password file <none>
I20260619 08:14:34.820978 6061 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:34.821049 6061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:34.821208 6061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:34.822338 6061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "e190e339cf73448dbe5549f44cf60c63"
format_stamp: "Formatted at 2026-06-19 08:14:34 on dist-test-slave-d0b5"
server_key: "ce29e037ab6add023750d9a6a8874414"
server_key_iv: "3204025fdec1e2159f898be2f9d506a7"
server_key_version: "encryptionkey@0"
I20260619 08:14:34.822700 6061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "e190e339cf73448dbe5549f44cf60c63"
format_stamp: "Formatted at 2026-06-19 08:14:34 on dist-test-slave-d0b5"
server_key: "ce29e037ab6add023750d9a6a8874414"
server_key_iv: "3204025fdec1e2159f898be2f9d506a7"
server_key_version: "encryptionkey@0"
I20260619 08:14:34.823997 6061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:34.824681 6076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:34.824864 6061 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:34.824968 6061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "e190e339cf73448dbe5549f44cf60c63"
format_stamp: "Formatted at 2026-06-19 08:14:34 on dist-test-slave-d0b5"
server_key: "ce29e037ab6add023750d9a6a8874414"
server_key_iv: "3204025fdec1e2159f898be2f9d506a7"
server_key_version: "encryptionkey@0"
I20260619 08:14:34.825039 6061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:34.846310 6061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:34.848731 6061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:34.848855 6061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:34.852854 6061 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:43555
I20260619 08:14:34.852867 6128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:43555 every 8 connection(s)
I20260619 08:14:34.853497 6061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:34.854223 6129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:34.856477 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63: Bootstrap starting.
I20260619 08:14:34.857054 6129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:34.857398 6129 log.cc:824] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:34.858081 6129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63: No bootstrap required, opened a new log
I20260619 08:14:34.859704 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6061
I20260619 08:14:34.859635 6129 raft_consensus.cc:348] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } }
I20260619 08:14:34.859787 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:34.859792 6129 raft_consensus.cc:374] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:34.859858 6129 raft_consensus.cc:729] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e190e339cf73448dbe5549f44cf60c63, State: Initialized, Role: FOLLOWER
I20260619 08:14:34.860008 2527 external_mini_cluster.cc:1428] Setting key e403ca1d8140f7281d7af38c82ad6e3e
I20260619 08:14:34.860013 6129 consensus_queue.cc:260] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } }
I20260619 08:14:34.860103 6129 raft_consensus.cc:388] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:34.860142 6129 raft_consensus.cc:482] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:34.860194 6129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:34.861490 6129 raft_consensus.cc:504] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } }
I20260619 08:14:34.861608 6129 leader_election.cc:302] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e190e339cf73448dbe5549f44cf60c63; no voters:
I20260619 08:14:34.861804 6129 leader_election.cc:288] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:34.861866 6134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:34.862001 6134 raft_consensus.cc:686] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [term 1 LEADER]: Becoming Leader. State: Replica: e190e339cf73448dbe5549f44cf60c63, State: Running, Role: LEADER
I20260619 08:14:34.862048 6129 sys_catalog.cc:564] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:34.862120 6134 consensus_queue.cc:237] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } }
I20260619 08:14:34.862433 6136 sys_catalog.cc:455] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e190e339cf73448dbe5549f44cf60c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } } }
I20260619 08:14:34.862553 6136 sys_catalog.cc:458] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:34.862440 6135 sys_catalog.cc:455] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e190e339cf73448dbe5549f44cf60c63. Latest consensus state: current_term: 1 leader_uuid: "e190e339cf73448dbe5549f44cf60c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e190e339cf73448dbe5549f44cf60c63" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43555 } } }
I20260619 08:14:34.862759 6135 sys_catalog.cc:458] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:34.862939 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:34.863600 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:34.865648 6142 catalog_manager.cc:1347] Generated new cluster ID: 8496dfa524c148f5872105cddf42641c
I20260619 08:14:34.865701 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:34 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:34.871613 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:34.860491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.2.119.254:43555)
0619 08:14:34.860688 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:34.860701 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:34.860742 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:14:34.860896 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:34.860905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:34.860988 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:34.861138 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:34.861827 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:34.863311 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:34.864506 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:34.865010 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:34.865130 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:34.867883 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:34.867887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:34.867887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:34.867900 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:34.869907 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:34.870330 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:34.870333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:34.870333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:34.870448 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:34.870672 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:34.870673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:34.870674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:34.870987 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:34.871122 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:34.871269 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:14:34.871404 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260619 08:14:34.883636 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:34.884405 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:34.888316 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e190e339cf73448dbe5549f44cf60c63: Generated new TSK 0
I20260619 08:14:34.888515 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:34.924558 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:35.005350 6155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:35.005551 6155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:35.005574 6155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:35.006733 6155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:35.006778 6155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:35.006804 6155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:35.008308 6155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:35.008559 6155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:35.008838 6155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:35.013876 6160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.013890 6161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.013859 6163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:35.014156 6155 server_base.cc:1034] running on GCE node
I20260619 08:14:35.014372 6155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:35.014585 6155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:35.015762 6155 hybrid_clock.cc:648] HybridClock initialized: now 1781856875015753 us; error 36 us; skew 500 ppm
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:35.018538 6155 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:35.019083 6155 webserver.cc:460] Webserver started at http://127.2.119.193:46641/ using document root <none> and password file <none>
I20260619 08:14:35.019332 6155 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:35.019400 6155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:35.019577 6155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:35.020757 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "2b670a01939540dfb4d411dc3656b520"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "8306a1ad48108c0a08483b1ec6c80c54"
server_key_iv: "ec8a9662a00ac0ac458003876e40c6c4"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.021165 6155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "2b670a01939540dfb4d411dc3656b520"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "8306a1ad48108c0a08483b1ec6c80c54"
server_key_iv: "ec8a9662a00ac0ac458003876e40c6c4"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.022580 6155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:35.023384 6170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.023591 6155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:35.023696 6155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "2b670a01939540dfb4d411dc3656b520"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "8306a1ad48108c0a08483b1ec6c80c54"
server_key_iv: "ec8a9662a00ac0ac458003876e40c6c4"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.023772 6155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:35.046423 6155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:35.049064 6155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:35.049256 6155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:35.049557 6155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:35.049947 6155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:35.050005 6155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.050040 6155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:35.050086 6155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.055902 6155 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:33729
I20260619 08:14:35.055954 6283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:33729 every 8 connection(s)
I20260619 08:14:35.056286 6155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:35.059593 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6155
I20260619 08:14:35.059689 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:35.059900 2527 external_mini_cluster.cc:1428] Setting key a92c8b87623aa62022621134ece2267e
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:35.066861 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.057009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54771 (local address 127.2.119.254:43555)
0619 08:14:35.057068 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:35.057070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:35.057312 (+ 242us) server_negotiation.cc:407] Connection header received
0619 08:14:35.058382 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:35.058384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:35.058401 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:35.058452 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:35.059079 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.059772 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.061023 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.061415 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.061489 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.063726 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:35.063751 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:35.063752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:35.063762 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:35.065510 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.065884 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.065885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.065886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.065951 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.066188 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.066189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.066190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.066335 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:35.066408 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:35.066666 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 08:14:35.066754 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:14:35.066879 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.057012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43555 (local address 127.2.119.193:54771)
0619 08:14:35.057186 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:35.057223 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:35.058259 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:35.058534 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:35.058538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:35.058788 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:35.058980 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.058984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.059875 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.059876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.060929 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.060931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.061507 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.061508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.061651 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.062228 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:35.062237 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:35.063637 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:35.065587 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.065593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.065595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.065806 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.066022 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.066023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.066023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.066120 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.066401 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:35.066405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:35.066529 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 08:14:35.066610 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:14:35.066722 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260619 08:14:35.067391 6284 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43555
I20260619 08:14:35.067487 6284 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:35.067621 6284 heartbeater.cc:511] Master 127.2.119.254:43555 requested a full tablet report, sending...
I20260619 08:14:35.068153 6093 ts_manager.cc:194] Registered new tserver with Master: 2b670a01939540dfb4d411dc3656b520 (127.2.119.193:33729)
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:14:35.068816 6093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:54771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:35.086799 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:35.166688 6289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:35.166880 6289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:35.166903 6289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:35.168027 6289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:35.168071 6289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:35.168097 6289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:35.169575 6289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:35.169816 6289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:35.170038 6289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:35.174877 6294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.174896 6297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.174896 6295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:35.175187 6289 server_base.cc:1034] running on GCE node
I20260619 08:14:35.175427 6289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:35.175652 6289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:35.176812 6289 hybrid_clock.cc:648] HybridClock initialized: now 1781856875176809 us; error 36 us; skew 500 ppm
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:35.179318 6289 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:35.179684 6289 webserver.cc:460] Webserver started at http://127.2.119.194:35851/ using document root <none> and password file <none>
I20260619 08:14:35.179852 6289 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:35.179899 6289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:35.179986 6289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:35.180928 6289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "eeb7bc1e732a429e8792b413af0e5127"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "c37820ecf83bf50448e92dc3f54bb56c"
server_key_iv: "fb2d6e3bdf99b06895a468c90596fe12"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.181277 6289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "eeb7bc1e732a429e8792b413af0e5127"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "c37820ecf83bf50448e92dc3f54bb56c"
server_key_iv: "fb2d6e3bdf99b06895a468c90596fe12"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.182569 6289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:35.183264 6304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.183442 6289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:35.183519 6289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "eeb7bc1e732a429e8792b413af0e5127"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "c37820ecf83bf50448e92dc3f54bb56c"
server_key_iv: "fb2d6e3bdf99b06895a468c90596fe12"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.183568 6289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:35.192811 6289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:35.195024 6289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:35.195129 6289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:35.195290 6289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:35.195528 6289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:35.195559 6289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.195580 6289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:35.195591 6289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.200943 6289 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:45223
I20260619 08:14:35.201002 6417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:45223 every 8 connection(s)
I20260619 08:14:35.201380 6289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:35.210403 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.202130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:38265 (local address 127.2.119.254:43555)
0619 08:14:35.202199 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:35.202201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:35.202405 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:14:35.203277 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:35.203278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:35.203287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:35.203341 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:35.203887 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.204380 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.205353 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.205569 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.205625 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.207678 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:35.207679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:35.207679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:35.207686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:35.209061 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.209454 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.209455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.209455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.209511 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.209773 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.209774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.209774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.209874 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:35.209932 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:35.210167 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:14:35.210270 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:14:35.210536 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.202099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43555 (local address 127.2.119.194:38265)
0619 08:14:35.202275 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:35.202308 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:35.203184 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:35.203396 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:35.203399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:35.203629 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:35.203808 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.203812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.204455 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.204456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.205220 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.205221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.205640 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.205640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.205738 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.206189 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:35.206197 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:35.207569 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:35.209137 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.209143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.209144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.209380 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.209599 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.209600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.209600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.209707 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.209931 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:35.209935 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:35.210059 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 08:14:35.210142 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:14:35.210275 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260619 08:14:35.211078 6418 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43555
I20260619 08:14:35.211177 6418 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:35.211355 6418 heartbeater.cc:511] Master 127.2.119.254:43555 requested a full tablet report, sending...
I20260619 08:14:35.211620 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6289
I20260619 08:14:35.211755 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:35.211803 6093 ts_manager.cc:194] Registered new tserver with Master: eeb7bc1e732a429e8792b413af0e5127 (127.2.119.194:45223)
I20260619 08:14:35.211938 2527 external_mini_cluster.cc:1428] Setting key e9520ac6d211df2e62c307e9df619f46
I20260619 08:14:35.212293 6093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:38265
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:35.236817 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:35.316118 6423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:35.316311 6423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:35.316334 6423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:35.317525 6423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:35.317567 6423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:35.317593 6423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:35.319054 6423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:35.319288 6423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:35.319525 6423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:35.324774 6428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.324777 6429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:35.324776 6431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:35.325063 6423 server_base.cc:1034] running on GCE node
I20260619 08:14:35.325372 6423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:35.325598 6423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:35.326767 6423 hybrid_clock.cc:648] HybridClock initialized: now 1781856875326756 us; error 26 us; skew 500 ppm
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:35.329536 6423 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:35.329919 6423 webserver.cc:460] Webserver started at http://127.2.119.195:33507/ using document root <none> and password file <none>
I20260619 08:14:35.330153 6423 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:35.330199 6423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:35.330322 6423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:35.331378 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "32da6b7e167341eb9247d4322c4671cc"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "e7e9fbed841d5890c98d1c00689aef34"
server_key_iv: "78478daee073152a4c3d65f0d93d0046"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.331761 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "32da6b7e167341eb9247d4322c4671cc"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "e7e9fbed841d5890c98d1c00689aef34"
server_key_iv: "78478daee073152a4c3d65f0d93d0046"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.333457 6423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:35.334298 6438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.334547 6423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:35.334647 6423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "32da6b7e167341eb9247d4322c4671cc"
format_stamp: "Formatted at 2026-06-19 08:14:35 on dist-test-slave-d0b5"
server_key: "e7e9fbed841d5890c98d1c00689aef34"
server_key_iv: "78478daee073152a4c3d65f0d93d0046"
server_key_version: "encryptionkey@0"
I20260619 08:14:35.334730 6423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:35.342481 6423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:35.344501 6423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:35.344609 6423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:35.344765 6423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:35.345125 6423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:35.345160 6423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.345181 6423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:35.345194 6423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:35.350852 6423 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44287
I20260619 08:14:35.350989 6551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44287 every 8 connection(s)
I20260619 08:14:35.351246 6423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:35.351576 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6423
I20260619 08:14:35.351684 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:35.351905 2527 external_mini_cluster.cc:1428] Setting key cdc3d1c7ae3772bae3a7362a42b0c51e
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:35.360906 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.352072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:56343 (local address 127.2.119.254:43555)
0619 08:14:35.352134 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:35.352137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:35.352368 (+ 231us) server_negotiation.cc:407] Connection header received
0619 08:14:35.353558 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:35.353559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:35.353571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:35.353628 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:35.354257 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.354768 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.355788 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.355996 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.356070 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.358258 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:35.358260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:35.358260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:35.358270 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:35.359509 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.359951 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.359953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.359953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.360019 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.360260 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.360261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.360262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.360383 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:35.360461 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:35.360727 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 08:14:35.360815 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:14:35.361093 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.351992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43555 (local address 127.2.119.195:56343)
0619 08:14:35.352217 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:35.352270 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:35.353452 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:35.353695 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:35.353700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:35.353969 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:35.354153 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.354157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.354862 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.354863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.355685 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:35.355687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.356076 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.356077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:35.356207 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.356658 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:35.356666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:35.358159 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:35.359604 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.359610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.359613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.359844 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.360086 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:35.360087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:35.360087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:35.360192 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:35.360451 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:35.360457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:35.360596 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 08:14:35.360705 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 08:14:35.360859 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
I20260619 08:14:35.361600 6552 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43555
I20260619 08:14:35.361681 6552 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:35.361887 6552 heartbeater.cc:511] Master 127.2.119.254:43555 requested a full tablet report, sending...
I20260619 08:14:35.362296 6092 ts_manager.cc:194] Registered new tserver with Master: 32da6b7e167341eb9247d4322c4671cc (127.2.119.195:44287)
I20260619 08:14:35.362720 6092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:56343
I20260619 08:14:35.363762 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:35.386325 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.378398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.2.119.193:33729)
0619 08:14:35.378585 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:35.378589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:35.378600 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:35.378624 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:35.378626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:35.378641 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:35.378706 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:35.379191 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.380007 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.380782 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.381103 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.381167 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.383454 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:35.383461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:35.383461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:35.383467 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:35.384990 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.385440 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.385443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.385443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.385509 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.385710 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.385710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.385710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.385885 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:35.386017 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:35.386170 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 08:14:35.386248 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:35.394605 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:35.387825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.2.119.254:43555)
0619 08:14:35.387867 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:35.387869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:35.387910 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:14:35.387931 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:35.387931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:35.387939 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:35.387979 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:35.388449 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.388985 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.389717 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:35.389914 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:35.389987 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:35.392232 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:35.392233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:35.392233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:35.392240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:35.393586 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.393873 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.393873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.393874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.393946 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:35.394170 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:35.394171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:35.394171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:35.394305 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:35.394393 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:35.394486 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 08:14:35.394543 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:14:35.394855 6092 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344
I20260619 08:14:35.395259 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6155
I20260619 08:14:35.399894 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6289
I20260619 08:14:35.404047 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6423
I20260619 08:14:35.407969 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6061
2026-06-19T08:14:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1836 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6564](info): set up 2 sockets
Jun 19 08:14:35 dist-test-slave-d0b5 krb5kdc[6564](info): commencing operation
krb5kdc: starting...
W20260619 08:14:36.446449 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:36.568576 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39747
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:39747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:36.650197 6577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:36.650432 6577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:36.650471 6577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:36.651713 6577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:36.651777 6577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:36.651801 6577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:36.651832 6577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:36.651861 6577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:36.653532 6577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:39747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:36.653811 6577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:36.654045 6577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:36.658838 6582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:36.658844 6583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:36.658838 6585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:36.659204 6577 server_base.cc:1034] running on GCE node
I20260619 08:14:36.659442 6577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:36.659719 6577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:36.660923 6577 hybrid_clock.cc:648] HybridClock initialized: now 1781856876660901 us; error 33 us; skew 500 ppm
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:36.663514 6577 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:36.663936 6577 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:36.664017 6577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:36.664194 6577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:36.665263 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "c30dce1c780c4e9d800ab1b3804606c3"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "08b53cfbafa2546e210964965fe2ac76"
server_key_iv: "984b9af5fcef4e99f4eee5e714df6f77"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.665685 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "c30dce1c780c4e9d800ab1b3804606c3"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "08b53cfbafa2546e210964965fe2ac76"
server_key_iv: "984b9af5fcef4e99f4eee5e714df6f77"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.667028 6577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:36.667737 6591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:36.667910 6577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:36.668006 6577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "c30dce1c780c4e9d800ab1b3804606c3"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "08b53cfbafa2546e210964965fe2ac76"
server_key_iv: "984b9af5fcef4e99f4eee5e714df6f77"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.668087 6577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:36.679780 6577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:36.681823 6577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:36.681991 6577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:36.686178 6577 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:39747
I20260619 08:14:36.686241 6643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:39747 every 8 connection(s)
I20260619 08:14:36.686544 6577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:36.687480 6644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:36.690292 6644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: Bootstrap starting.
I20260619 08:14:36.690876 6644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:36.691179 6644 log.cc:824] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:36.691836 6644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: No bootstrap required, opened a new log
I20260619 08:14:36.693467 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6577
I20260619 08:14:36.693576 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:36.693624 6644 raft_consensus.cc:348] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } }
I20260619 08:14:36.693768 2527 external_mini_cluster.cc:1428] Setting key 229f16d185887e440b234ebc75c8865c
I20260619 08:14:36.693804 6644 raft_consensus.cc:374] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:36.693845 6644 raft_consensus.cc:729] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30dce1c780c4e9d800ab1b3804606c3, State: Initialized, Role: FOLLOWER
I20260619 08:14:36.693993 6644 consensus_queue.cc:260] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } }
I20260619 08:14:36.694084 6644 raft_consensus.cc:388] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:36.694120 6644 raft_consensus.cc:482] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:36.694172 6644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:36.695335 6644 raft_consensus.cc:504] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } }
I20260619 08:14:36.695459 6644 leader_election.cc:302] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c30dce1c780c4e9d800ab1b3804606c3; no voters:
I20260619 08:14:36.695611 6644 leader_election.cc:288] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:36.695652 6649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:36.695811 6649 raft_consensus.cc:686] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [term 1 LEADER]: Becoming Leader. State: Replica: c30dce1c780c4e9d800ab1b3804606c3, State: Running, Role: LEADER
I20260619 08:14:36.695914 6644 sys_catalog.cc:564] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:36.695909 6649 consensus_queue.cc:237] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } }
I20260619 08:14:36.697692 6650 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c30dce1c780c4e9d800ab1b3804606c3. Latest consensus state: current_term: 1 leader_uuid: "c30dce1c780c4e9d800ab1b3804606c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } } }
I20260619 08:14:36.698158 6650 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:36.697836 6651 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c30dce1c780c4e9d800ab1b3804606c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30dce1c780c4e9d800ab1b3804606c3" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39747 } } }
I20260619 08:14:36.698323 6651 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:36.698432 6665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:36.698886 6665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:36.700567 6665 catalog_manager.cc:1347] Generated new cluster ID: 5d6fb76923b14ca6a9d47bb478b1acce
I20260619 08:14:36.700623 6665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:36.705538 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:36.694240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.2.119.254:39747)
0619 08:14:36.694477 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:36.694485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:36.694521 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:14:36.694629 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:36.694634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:36.694684 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:36.694812 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:36.695636 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.697191 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.698133 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.698774 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.698893 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:36.701584 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:36.701588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:36.701588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:36.701600 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:36.703762 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:36.704157 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:36.704161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:36.704161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:36.704307 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:36.704513 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:36.704513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:36.704514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:36.704852 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:36.705031 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:36.705104 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:14:36.705256 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":110,"threads_started":1}
I20260619 08:14:36.708348 6665 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:36.708902 6665 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:36.711767 6665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: Generated new TSK 0
I20260619 08:14:36.711984 6665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:36.746150 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:36.830788 6670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:36.831012 6670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:36.831049 6670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:36.832170 6670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:36.832231 6670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:36.832278 6670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:36.833913 6670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:36.834169 6670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:36.834395 6670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:36.840543 6676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:36.840552 6675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:36.840865 6670 server_base.cc:1034] running on GCE node
W20260619 08:14:36.840994 6678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:36.841223 6670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:36.841562 6670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:36.842746 6670 hybrid_clock.cc:648] HybridClock initialized: now 1781856876842730 us; error 32 us; skew 500 ppm
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:36.850239 6670 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:36.850631 6670 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:36.850682 6670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:36.850829 6670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:36.851994 6670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "1545f9ab03384513a8adb2015503c2d7"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "43c21b48b1bc7ca0c9226ed7fffadf0d"
server_key_iv: "9320c55a33d97234f64aa349d093296c"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.852352 6670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "1545f9ab03384513a8adb2015503c2d7"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "43c21b48b1bc7ca0c9226ed7fffadf0d"
server_key_iv: "9320c55a33d97234f64aa349d093296c"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.853785 6670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:36.854554 6684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:36.854710 6670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:36.854833 6670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "1545f9ab03384513a8adb2015503c2d7"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "43c21b48b1bc7ca0c9226ed7fffadf0d"
server_key_iv: "9320c55a33d97234f64aa349d093296c"
server_key_version: "encryptionkey@0"
I20260619 08:14:36.854910 6670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:36.868917 6670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:36.871320 6670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:36.871508 6670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:36.871745 6670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:36.872076 6670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:36.872138 6670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:36.872179 6670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:36.872205 6670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:36.878127 6670 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:39135
I20260619 08:14:36.878180 6797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:39135 every 8 connection(s)
I20260619 08:14:36.878427 6670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:36.881474 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6670
I20260619 08:14:36.881582 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:36.881769 2527 external_mini_cluster.cc:1428] Setting key 69e831629b96568ae30844fdd5d0f527
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:36.888681 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:36.879130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:57779 (local address 127.2.119.254:39747)
0619 08:14:36.879214 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:36.879217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:36.879373 (+ 156us) server_negotiation.cc:407] Connection header received
0619 08:14:36.879821 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:36.879823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:36.879836 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:36.879887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:36.880761 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.881310 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.882727 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.882922 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.882988 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:36.885560 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:36.885562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:36.885563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:36.885571 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:36.887270 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:36.887664 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:36.887665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:36.887666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:36.887732 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:36.887980 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:36.887982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:36.887982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:36.888137 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:36.888217 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:36.888465 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 08:14:36.888554 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 08:14:36.888710 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:36.879091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39747 (local address 127.2.119.193:57779)
0619 08:14:36.879257 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:36.879292 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:36.879744 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:36.879944 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:36.879947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:36.880174 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:36.880668 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:36.880673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.881439 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.881441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:36.882646 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:36.882650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:36.883020 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:36.883021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:36.883177 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:36.883655 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:36.883666 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:36.885471 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:36.887352 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:36.887358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:36.887360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:36.887586 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:36.887805 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:36.887806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:36.887806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:36.887904 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:36.888209 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:36.888213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:36.888328 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 08:14:36.888410 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:14:36.888549 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":108,"threads_started":1}
I20260619 08:14:36.889241 6798 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39747
I20260619 08:14:36.889377 6798 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:36.889532 6798 heartbeater.cc:511] Master 127.2.119.254:39747 requested a full tablet report, sending...
I20260619 08:14:36.890060 6608 ts_manager.cc:194] Registered new tserver with Master: 1545f9ab03384513a8adb2015503c2d7 (127.2.119.193:39135)
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:14:36.890658 6608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:57779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:36.907390 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:36.987076 6803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:36.987303 6803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:36.987329 6803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:36.988545 6803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:36.988590 6803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:36.988617 6803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:36.990202 6803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:36.990450 6803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:36.990676 6803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:36.995731 6808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:36.995784 6803 server_base.cc:1034] running on GCE node
W20260619 08:14:36.995731 6811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:36.995750 6809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:36.996215 6803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:36.996487 6803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:36.997634 6803 hybrid_clock.cc:648] HybridClock initialized: now 1781856876997622 us; error 29 us; skew 500 ppm
Jun 19 08:14:36 dist-test-slave-d0b5 krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:37.000156 6803 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:37.000494 6803 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.000540 6803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.000702 6803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.001786 6803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "4fb73a3deb8e4041bc2fe3c90eecb211"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "83d64106ead80f96de6a43d9f98fd052"
server_key_iv: "2aec01ad3064c76d9a79149635e85e68"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.002130 6803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "4fb73a3deb8e4041bc2fe3c90eecb211"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "83d64106ead80f96de6a43d9f98fd052"
server_key_iv: "2aec01ad3064c76d9a79149635e85e68"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.003360 6803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:37.004063 6817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.004261 6803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:37.004370 6803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "4fb73a3deb8e4041bc2fe3c90eecb211"
format_stamp: "Formatted at 2026-06-19 08:14:36 on dist-test-slave-d0b5"
server_key: "83d64106ead80f96de6a43d9f98fd052"
server_key_iv: "2aec01ad3064c76d9a79149635e85e68"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.004438 6803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.013880 6803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.016134 6803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.016278 6803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.016515 6803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:37.016834 6803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:37.016891 6803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.016930 6803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:37.016959 6803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.022833 6803 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:43723
I20260619 08:14:37.022859 6930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:43723 every 8 connection(s)
I20260619 08:14:37.023136 6803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:14:37 dist-test-slave-d0b5 krb5kdc[6564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856876, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:37.032121 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6803
I20260619 08:14:37.032254 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:37.032470 2527 external_mini_cluster.cc:1428] Setting key a9fc6b2cc0f225bcf44069f3d3a5fa78
I20260619 08:14:37.032759 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.023856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:43847 (local address 127.2.119.254:39747)
0619 08:14:37.023904 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.023906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.024068 (+ 162us) server_negotiation.cc:407] Connection header received
0619 08:14:37.024627 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.024628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.024638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.024704 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.025633 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.026128 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.027153 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.027336 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.027391 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.029677 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.029678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.029678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:37.029689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.031180 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.031664 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.031666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.031666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.031730 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.031993 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.031994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.031994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.032116 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.032220 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.032559 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.032653 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:37.032927 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.023813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39747 (local address 127.2.119.194:43847)
0619 08:14:37.023960 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.024001 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.024527 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.024758 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.024762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.025010 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.025555 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.025560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.026214 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.026215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.027078 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.027080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.027437 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.027439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.027552 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.028155 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:37.028163 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.029570 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.031315 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:37.031323 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:37.031325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:37.031580 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:37.031802 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:37.031803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:37.031803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:37.031913 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:37.032203 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.032210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.032413 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.032528 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.032688 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 08:14:37.033401 6931 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39747
I20260619 08:14:37.033497 6931 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:37.033663 6931 heartbeater.cc:511] Master 127.2.119.254:39747 requested a full tablet report, sending...
I20260619 08:14:37.034029 6608 ts_manager.cc:194] Registered new tserver with Master: 4fb73a3deb8e4041bc2fe3c90eecb211 (127.2.119.194:43723)
I20260619 08:14:37.034456 6608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:43847
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:37.056787 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:37.138527 6936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:37.138744 6936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:37.138787 6936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:37.140627 6936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:37.140683 6936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:37.140728 6936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:37.142370 6936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:37.142567 6936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:37.142740 6936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:37.147850 6941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.147847 6942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.147895 6936 server_base.cc:1034] running on GCE node
W20260619 08:14:37.147847 6944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.148350 6936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:37.148613 6936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:37.149806 6936 hybrid_clock.cc:648] HybridClock initialized: now 1781856877149789 us; error 27 us; skew 500 ppm
Jun 19 08:14:37 dist-test-slave-d0b5 krb5kdc[6564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856877, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:37.152688 6936 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:37.153106 6936 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.153189 6936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.153390 6936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.154599 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "651480ec975949829da39d5c9c658cff"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "4beea3f8f7b523908ba2ab32547d9c2d"
server_key_iv: "56b2eb57d4c3710d3d0d6a99bee4f2f8"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.155007 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "651480ec975949829da39d5c9c658cff"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "4beea3f8f7b523908ba2ab32547d9c2d"
server_key_iv: "56b2eb57d4c3710d3d0d6a99bee4f2f8"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.156342 6936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:37.157115 6950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.157339 6936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:37.157454 6936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "651480ec975949829da39d5c9c658cff"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "4beea3f8f7b523908ba2ab32547d9c2d"
server_key_iv: "56b2eb57d4c3710d3d0d6a99bee4f2f8"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.157541 6936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.170859 6936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.173458 6936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.173578 6936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.173766 6936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:37.174104 6936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:37.174144 6936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.174167 6936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:37.174220 6936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.180377 6936 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44731
I20260619 08:14:37.180409 7063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44731 every 8 connection(s)
I20260619 08:14:37.180711 6936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:37.181521 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 6936
I20260619 08:14:37.181641 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:37.181887 2527 external_mini_cluster.cc:1428] Setting key 61c489d2dd9f09baa18881187e57b607
Jun 19 08:14:37 dist-test-slave-d0b5 krb5kdc[6564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856877, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:37.190171 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.181545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:39689 (local address 127.2.119.254:39747)
0619 08:14:37.181615 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.181617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.181980 (+ 363us) server_negotiation.cc:407] Connection header received
0619 08:14:37.182627 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.182627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.182637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.182669 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.183704 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.184187 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.185209 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.185468 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.185525 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.187691 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.187692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.187693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:37.187699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.188897 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.189284 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.189285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.189285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.189392 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.189622 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.189623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.189623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.189722 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.189771 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.190052 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.190108 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:37.190366 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.181530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39747 (local address 127.2.119.195:39689)
0619 08:14:37.181773 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.181830 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.182505 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.182761 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.182765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.183119 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.183620 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.183625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.184273 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.184273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.185131 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.185133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.185542 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.185543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.185648 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.186208 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:37.186217 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.187606 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.188974 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:37.188979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:37.188981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:37.189211 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:37.189458 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:37.189458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:37.189458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:37.189561 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:37.189793 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.189797 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.189936 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.190025 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.190146 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260619 08:14:37.190868 7064 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39747
I20260619 08:14:37.191000 7064 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:37.191144 7064 heartbeater.cc:511] Master 127.2.119.254:39747 requested a full tablet report, sending...
I20260619 08:14:37.191576 6608 ts_manager.cc:194] Registered new tserver with Master: 651480ec975949829da39d5c9c658cff (127.2.119.195:44731)
I20260619 08:14:37.191959 6608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:39689
I20260619 08:14:37.193645 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:37.200695 6648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.194910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.2.119.254:39747)
0619 08:14:37.194994 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.194996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.195117 (+ 121us) server_negotiation.cc:407] Connection header received
0619 08:14:37.195170 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.195171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.195178 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.195225 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.195777 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.196273 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.197088 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.197262 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.197380 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.198616 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.198617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.198617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:37.198622 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.199729 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.200032 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.200032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.200033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.200078 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:37.200236 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:37.200236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:37.200237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:37.200355 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.200428 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.200533 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.200587 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 08:14:37.202255 6608 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:37.203814 6608 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:37.210731 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.205964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39135 (local address 127.0.0.1:34086)
0619 08:14:37.206061 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.206100 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.206158 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.206746 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.206749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.206751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:37.206819 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.206821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.208176 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.208177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.209465 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.209468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.210396 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.210398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.210510 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.210526 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.210574 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.210633 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":12,"thread_start_us":53,"threads_started":1}
I20260619 08:14:37.210733 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.205974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:43723 (local address 127.0.0.1:45480)
0619 08:14:37.206407 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.206418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.206485 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.206954 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.206955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.206956 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:37.206992 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.206993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.207970 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.207971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.209465 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.209468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.210454 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.210454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.210508 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.210525 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.210602 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.210637 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":67,"thread_start_us":57,"threads_started":1}
I20260619 08:14:37.210759 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.206172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.2.119.194:43723)
0619 08:14:37.206354 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.206359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.206487 (+ 128us) server_negotiation.cc:407] Connection header received
0619 08:14:37.206556 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.206558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.206577 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.206661 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:37.207118 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.207873 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.209556 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.210290 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.210366 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.210425 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.210630 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.210672 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260619 08:14:37.211140 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.206392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.2.119.193:39135)
0619 08:14:37.206562 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.206566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.206575 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:37.206613 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.206614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.206629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.206735 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:37.206906 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.208087 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.209558 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.210290 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.210928 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.210995 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.211019 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.211057 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260619 08:14:37.211578 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.206163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44731 (local address 127.0.0.1:44486)
0619 08:14:37.206578 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.206587 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.206663 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.206857 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.206858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.206860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:37.206914 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.206916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.209566 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.209567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.210231 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.210232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.211352 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.211354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.211416 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.211419 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.211473 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.211510 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":46,"threads_started":1}
I20260619 08:14:37.211596 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.206262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.2.119.195:44731)
0619 08:14:37.206420 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.206425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.206715 (+ 290us) server_negotiation.cc:407] Connection header received
0619 08:14:37.206755 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.206757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.206773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.206847 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:37.207018 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.208089 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.210328 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.211277 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.211339 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.211397 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.211488 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.211523 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260619 08:14:37.211794 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb (DEFAULT_TABLE table=test-table [id=c67701121cfc4e52a5ee4a401a398a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.211796 6865 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb (DEFAULT_TABLE table=test-table [id=c67701121cfc4e52a5ee4a401a398a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.212137 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2fe5fe792e483a81aa9ac42204eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.212136 6865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2fe5fe792e483a81aa9ac42204eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.213836 6998 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb (DEFAULT_TABLE table=test-table [id=c67701121cfc4e52a5ee4a401a398a95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.214134 6998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2fe5fe792e483a81aa9ac42204eeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.215596 7079 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: Bootstrap starting.
I20260619 08:14:37.216249 7079 tablet_bootstrap.cc:654] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.216892 7079 log.cc:824] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.217044 7080 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Bootstrap starting.
I20260619 08:14:37.217408 7078 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Bootstrap starting.
I20260619 08:14:37.217720 7080 tablet_bootstrap.cc:654] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.217844 7079 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: No bootstrap required, opened a new log
I20260619 08:14:37.217922 7079 ts_tablet_manager.cc:1397] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:37.218011 7078 tablet_bootstrap.cc:654] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.218348 7080 log.cc:824] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.218632 7078 log.cc:824] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.219277 7078 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: No bootstrap required, opened a new log
I20260619 08:14:37.219396 7078 ts_tablet_manager.cc:1397] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:14:37.219696 7079 raft_consensus.cc:348] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.219821 7079 raft_consensus.cc:374] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.219843 7079 raft_consensus.cc:729] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1545f9ab03384513a8adb2015503c2d7, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.219981 7079 consensus_queue.cc:260] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.220239 7079 ts_tablet_manager.cc:1428] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:14:37.220280 6798 heartbeater.cc:503] Master 127.2.119.254:39747 was elected leader, sending a full tablet report...
I20260619 08:14:37.220778 7080 tablet_bootstrap.cc:492] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: No bootstrap required, opened a new log
I20260619 08:14:37.220825 7080 ts_tablet_manager.cc:1397] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:14:37.221719 7078 raft_consensus.cc:348] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.221872 7078 raft_consensus.cc:374] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.221915 7078 raft_consensus.cc:729] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fb73a3deb8e4041bc2fe3c90eecb211, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.222055 7078 consensus_queue.cc:260] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.222406 7078 ts_tablet_manager.cc:1428] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:14:37.222436 6931 heartbeater.cc:503] Master 127.2.119.254:39747 was elected leader, sending a full tablet report...
I20260619 08:14:37.222610 7080 raft_consensus.cc:348] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.222772 7080 raft_consensus.cc:374] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.222821 7080 raft_consensus.cc:729] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 651480ec975949829da39d5c9c658cff, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.222930 7080 consensus_queue.cc:260] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.223199 7064 heartbeater.cc:503] Master 127.2.119.254:39747 was elected leader, sending a full tablet report...
I20260619 08:14:37.223227 7080 ts_tablet_manager.cc:1428] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:14:37.223733 7084 raft_consensus.cc:482] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:37.223837 7084 raft_consensus.cc:504] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.224170 7084 leader_election.cc:288] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fb73a3deb8e4041bc2fe3c90eecb211 (127.2.119.194:43723), 651480ec975949829da39d5c9c658cff (127.2.119.195:44731)
I20260619 08:14:37.227106 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.224324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:55513 (local address 127.2.119.194:43723)
0619 08:14:37.224365 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.224367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.224513 (+ 146us) server_negotiation.cc:407] Connection header received
0619 08:14:37.224614 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.224615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.224623 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.224678 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:37.224827 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.225274 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226403 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.226765 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226848 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.226857 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.227013 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.227046 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:14:37.227188 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.224430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46149 (local address 127.2.119.195:44731)
0619 08:14:37.224488 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.224490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.224498 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:37.224575 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.224576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.224593 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.224650 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:37.224811 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.225273 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226392 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.226832 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226913 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.226921 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.227092 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.227124 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:37.227210 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.224323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44731 (local address 127.2.119.193:46149)
0619 08:14:37.224454 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.224466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.224510 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.224667 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.224669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.224671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:37.224731 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.224732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.225405 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.225406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.226274 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.226277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.226910 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.227009 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.227011 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.227085 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.227123 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":30,"thread_start_us":88,"threads_started":1}
I20260619 08:14:37.227118 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.224287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:43723 (local address 127.2.119.193:55513)
0619 08:14:37.224455 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.224466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.224535 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.224681 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.224682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.224683 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:37.224749 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.224750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.225417 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.225417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.226323 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:37.226326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.226847 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.226848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:37.226919 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.226922 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.227005 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.227045 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260619 08:14:37.227557 7018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2fe5fe792e483a81aa9ac42204eeeb" candidate_uuid: "1545f9ab03384513a8adb2015503c2d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "651480ec975949829da39d5c9c658cff" is_pre_election: true
I20260619 08:14:37.227689 7018 raft_consensus.cc:2454] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1545f9ab03384513a8adb2015503c2d7 in term 0.
I20260619 08:14:37.227846 6885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2fe5fe792e483a81aa9ac42204eeeb" candidate_uuid: "1545f9ab03384513a8adb2015503c2d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" is_pre_election: true
I20260619 08:14:37.227892 6686 leader_election.cc:302] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1545f9ab03384513a8adb2015503c2d7, 651480ec975949829da39d5c9c658cff; no voters:
I20260619 08:14:37.227998 6885 raft_consensus.cc:2454] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1545f9ab03384513a8adb2015503c2d7 in term 0.
I20260619 08:14:37.228051 7084 raft_consensus.cc:2781] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:37.228111 7084 raft_consensus.cc:482] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:37.228134 7084 raft_consensus.cc:3037] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.228746 7084 raft_consensus.cc:504] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.228951 7084 leader_election.cc:288] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [CANDIDATE]: Term 1 election: Requested vote from peers 4fb73a3deb8e4041bc2fe3c90eecb211 (127.2.119.194:43723), 651480ec975949829da39d5c9c658cff (127.2.119.195:44731)
I20260619 08:14:37.229009 6885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2fe5fe792e483a81aa9ac42204eeeb" candidate_uuid: "1545f9ab03384513a8adb2015503c2d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211"
I20260619 08:14:37.229059 7018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2fe5fe792e483a81aa9ac42204eeeb" candidate_uuid: "1545f9ab03384513a8adb2015503c2d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "651480ec975949829da39d5c9c658cff"
I20260619 08:14:37.229085 6885 raft_consensus.cc:3037] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.229118 7018 raft_consensus.cc:3037] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.229976 7018 raft_consensus.cc:2454] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1545f9ab03384513a8adb2015503c2d7 in term 1.
I20260619 08:14:37.229976 6885 raft_consensus.cc:2454] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1545f9ab03384513a8adb2015503c2d7 in term 1.
I20260619 08:14:37.230150 6686 leader_election.cc:302] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1545f9ab03384513a8adb2015503c2d7, 651480ec975949829da39d5c9c658cff; no voters:
I20260619 08:14:37.230305 7084 raft_consensus.cc:2781] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:37.230475 7084 raft_consensus.cc:686] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1545f9ab03384513a8adb2015503c2d7, State: Running, Role: LEADER
I20260619 08:14:37.230574 7084 consensus_queue.cc:237] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } }
I20260619 08:14:37.231405 6608 catalog_manager.cc:5495] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1545f9ab03384513a8adb2015503c2d7 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "1545f9ab03384513a8adb2015503c2d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1545f9ab03384513a8adb2015503c2d7" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:37.241724 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.239529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.2.119.193:39135)
0619 08:14:37.239581 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.239582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.239588 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:14:37.239614 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.239614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.239629 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.239681 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:37.239822 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.240273 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.241010 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:37.241188 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:37.241239 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:37.241407 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:37.241501 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:37.241564 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.241593 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.241610 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:14:37.243528 6885 raft_consensus.cc:1261] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 1 FOLLOWER]: Refusing update from remote peer 1545f9ab03384513a8adb2015503c2d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:37.243561 7018 raft_consensus.cc:1261] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 1 FOLLOWER]: Refusing update from remote peer 1545f9ab03384513a8adb2015503c2d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:37.243899 7088 consensus_queue.cc:1035] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fb73a3deb8e4041bc2fe3c90eecb211" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 43723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:37.244046 7084 consensus_queue.cc:1035] T 3b2fe5fe792e483a81aa9ac42204eeeb P 1545f9ab03384513a8adb2015503c2d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "651480ec975949829da39d5c9c658cff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:37.246656 7092 mvcc.cc:205] Tried to move back new op lower bound from 7298485769186603008 to 7298485769137078272. Current Snapshot: MvccSnapshot[applied={T|T < 7298485769186603008}]
I20260619 08:14:37.247033 7093 mvcc.cc:205] Tried to move back new op lower bound from 7298485769186603008 to 7298485769137078272. Current Snapshot: MvccSnapshot[applied={T|T < 7298485769186603008 or (T in {7298485769186603008})}]
I20260619 08:14:37.249574 6606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:37.249683 6606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:37.250434 6606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c30dce1c780c4e9d800ab1b3804606c3: Sending DeleteTablet for 3 replicas of tablet 3b2fe5fe792e483a81aa9ac42204eeeb
I20260619 08:14:37.250772 6865 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:45480
I20260619 08:14:37.250772 6998 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:44486
I20260619 08:14:37.250993 7101 tablet_replica.cc:330] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: stopping tablet replica
I20260619 08:14:37.251014 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6670
I20260619 08:14:37.251062 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2fe5fe792e483a81aa9ac42204eeeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:34086
I20260619 08:14:37.251091 7101 raft_consensus.cc:2229] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:37.251186 7101 raft_consensus.cc:2258] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:37.251821 7101 ts_tablet_manager.cc:1905] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:37.253299 7102 tablet_replica.cc:330] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: stopping tablet replica
I20260619 08:14:37.253342 7101 ts_tablet_manager.cc:1918] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:37.253434 7102 raft_consensus.cc:2229] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:37.253504 7101 log.cc:1196] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/3b2fe5fe792e483a81aa9ac42204eeeb
I20260619 08:14:37.253536 7102 raft_consensus.cc:2258] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:37.253798 7101 ts_tablet_manager.cc:1939] T 3b2fe5fe792e483a81aa9ac42204eeeb P 651480ec975949829da39d5c9c658cff: Deleting consensus metadata
I20260619 08:14:37.253969 7102 ts_tablet_manager.cc:1905] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:37.254207 6593 catalog_manager.cc:4841] TS 651480ec975949829da39d5c9c658cff (127.2.119.195:44731): tablet 3b2fe5fe792e483a81aa9ac42204eeeb (table test-table [id=c67701121cfc4e52a5ee4a401a398a95]) successfully deleted
I20260619 08:14:37.255378 7102 ts_tablet_manager.cc:1918] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:37.255435 7102 log.cc:1196] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/3b2fe5fe792e483a81aa9ac42204eeeb
I20260619 08:14:37.255697 7102 ts_tablet_manager.cc:1939] T 3b2fe5fe792e483a81aa9ac42204eeeb P 4fb73a3deb8e4041bc2fe3c90eecb211: Deleting consensus metadata
I20260619 08:14:37.256098 6592 catalog_manager.cc:4841] TS 4fb73a3deb8e4041bc2fe3c90eecb211 (127.2.119.194:43723): tablet 3b2fe5fe792e483a81aa9ac42204eeeb (table test-table [id=c67701121cfc4e52a5ee4a401a398a95]) successfully deleted
W20260619 08:14:37.256750 6593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:39135 (error 108)
I20260619 08:14:37.257218 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6803
I20260619 08:14:37.257265 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.257143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39135 (local address 127.0.0.1:34108)
0619 08:14:37.257197 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.257239 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:14:37.258385 6593 catalog_manager.cc:4584] TS 1545f9ab03384513a8adb2015503c2d7 (127.2.119.193:39135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b2fe5fe792e483a81aa9ac42204eeeb: Network error: Client connection negotiation failed: client connection to 127.2.119.193:39135: connect: Connection refused (error 111)
I20260619 08:14:37.261787 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6936
I20260619 08:14:37.266158 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 6577
2026-06-19T08:14:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1858 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T08:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:37Z Disabled control of system clock
I20260619 08:14:37.293737 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45271
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:14:37.379801 7107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:37.380105 7107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:37.380183 7107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:37.382198 7107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:37.382268 7107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:37.382289 7107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:37.382308 7107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:37.382324 7107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:37.384582 7107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:37.384902 7107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:37.385251 7107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:37.392544 7113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.392552 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.392686 7115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.392781 7107 server_base.cc:1034] running on GCE node
I20260619 08:14:37.393090 7107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:37.393437 7107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:37.394613 7107 hybrid_clock.cc:648] HybridClock initialized: now 1781856877394589 us; error 35 us; skew 500 ppm
I20260619 08:14:37.396209 7107 webserver.cc:460] Webserver started at http://127.2.119.254:38687/ using document root <none> and password file <none>
I20260619 08:14:37.396497 7107 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.396569 7107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.396732 7107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.398305 7107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "3edba6047c1345899baff54d7923bff4"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "02b55ba9b9776b4c96c13075b81c563d"
server_key_iv: "46c808cea075f7a116f0a5fa53b7a1f6"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.398756 7107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "3edba6047c1345899baff54d7923bff4"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "02b55ba9b9776b4c96c13075b81c563d"
server_key_iv: "46c808cea075f7a116f0a5fa53b7a1f6"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.400400 7107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:37.401379 7121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.401568 7107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:37.401662 7107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "3edba6047c1345899baff54d7923bff4"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "02b55ba9b9776b4c96c13075b81c563d"
server_key_iv: "46c808cea075f7a116f0a5fa53b7a1f6"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.401734 7107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.407238 7107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.407560 7107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.407691 7107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.412911 7107 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45271
I20260619 08:14:37.412971 7173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45271 every 8 connection(s)
I20260619 08:14:37.413364 7107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:37.414275 7174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.418359 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7107
I20260619 08:14:37.418473 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:37.418666 2527 external_mini_cluster.cc:1428] Setting key 289f7183935d4166bceb1a5f92367c17
I20260619 08:14:37.418907 7174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: Bootstrap starting.
I20260619 08:14:37.419634 7174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.419987 7174 log.cc:824] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.420856 7174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: No bootstrap required, opened a new log
I20260619 08:14:37.421592 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.419182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38506 (local address 127.2.119.254:45271)
0619 08:14:37.419385 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.419394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.419428 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:14:37.420068 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.420077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.420135 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.420288 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.421026 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.421029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.421030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.421034 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.421088 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.421099 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.421193 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.421256 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.421445 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260619 08:14:37.423331 7174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } }
I20260619 08:14:37.423542 7174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.423584 7174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3edba6047c1345899baff54d7923bff4, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.423744 7174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } }
I20260619 08:14:37.424263 7174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:37.424317 7174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:37.424386 7174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.425189 7174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } }
I20260619 08:14:37.425346 7174 leader_election.cc:302] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3edba6047c1345899baff54d7923bff4; no voters:
I20260619 08:14:37.425549 7174 leader_election.cc:288] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:37.425580 7179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:37.425738 7179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [term 1 LEADER]: Becoming Leader. State: Replica: 3edba6047c1345899baff54d7923bff4, State: Running, Role: LEADER
I20260619 08:14:37.425841 7179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } }
I20260619 08:14:37.425858 7174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:37.426193 7181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3edba6047c1345899baff54d7923bff4. Latest consensus state: current_term: 1 leader_uuid: "3edba6047c1345899baff54d7923bff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } } }
I20260619 08:14:37.426307 7181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:37.426189 7180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3edba6047c1345899baff54d7923bff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3edba6047c1345899baff54d7923bff4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45271 } } }
I20260619 08:14:37.426465 7180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:37.426978 7185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:37.427495 7185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:37.429075 7185 catalog_manager.cc:1347] Generated new cluster ID: 3283f37880bb4f6fa8127c85d58760a7
I20260619 08:14:37.429123 7185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:37.441675 7185 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:37.441849 7185 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:37.446959 7185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: Generated new TSK 0
I20260619 08:14:37.447183 7185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:14:37.452244 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--builtin_ntp_servers=127.2.119.212:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:14:37.539561 7198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:37.539904 7198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:37.539934 7198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:37.541235 7198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:37.541283 7198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:37.541390 7198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:37.542863 7198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:37.543126 7198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:37.543471 7198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:37.548815 7203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.548792 7204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.548997 7198 server_base.cc:1034] running on GCE node
W20260619 08:14:37.549091 7206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.549356 7198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:37.549649 7198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:37.550843 7198 hybrid_clock.cc:648] HybridClock initialized: now 1781856877550824 us; error 32 us; skew 500 ppm
I20260619 08:14:37.552256 7198 webserver.cc:460] Webserver started at http://127.2.119.193:34891/ using document root <none> and password file <none>
I20260619 08:14:37.552588 7198 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.552667 7198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.552887 7198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.554504 7198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "299cd8cfd3e04f6199460d9ea935be8d"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "269346c16543e11fddf2a800e7277f8b"
server_key_iv: "73995822ea2fb168311e3964a0a4d70c"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.555043 7198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "299cd8cfd3e04f6199460d9ea935be8d"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "269346c16543e11fddf2a800e7277f8b"
server_key_iv: "73995822ea2fb168311e3964a0a4d70c"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.556519 7198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:14:37.557463 7212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.557651 7198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:37.557747 7198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "299cd8cfd3e04f6199460d9ea935be8d"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "269346c16543e11fddf2a800e7277f8b"
server_key_iv: "73995822ea2fb168311e3964a0a4d70c"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.557842 7198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.561386 7198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.561693 7198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.561842 7198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.562058 7198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:37.562373 7198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:37.562434 7198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.562476 7198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:37.562502 7198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.568252 7198 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:41499
I20260619 08:14:37.568302 7325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:41499 every 8 connection(s)
I20260619 08:14:37.568665 7198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:37.571871 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.569500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:47295 (local address 127.2.119.254:45271)
0619 08:14:37.569546 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.569548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.569774 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:14:37.570288 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.570290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.570296 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.570356 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.571600 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.571601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.571601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.571607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.571622 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.571627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.571681 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.571783 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.571799 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:37.572109 7328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.569482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45271 (local address 127.2.119.193:47295)
0619 08:14:37.569654 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.569687 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.570181 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.570430 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.570442 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.570451 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.571422 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.571422 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.571459 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.571489 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.571688 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.571693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.571696 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.571794 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.571858 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260619 08:14:37.572609 7326 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45271
I20260619 08:14:37.572702 7326 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:37.572813 7326 heartbeater.cc:511] Master 127.2.119.254:45271 requested a full tablet report, sending...
I20260619 08:14:37.573350 7138 ts_manager.cc:194] Registered new tserver with Master: 299cd8cfd3e04f6199460d9ea935be8d (127.2.119.193:41499)
I20260619 08:14:37.577543 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7198
I20260619 08:14:37.577656 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:37.577853 2527 external_mini_cluster.cc:1428] Setting key 0cb96ceb4f69cb35f7d8822acd0d55a1
I20260619 08:14:37.578809 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--builtin_ntp_servers=127.2.119.212:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:14:37.662027 7329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:37.662240 7329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:37.662264 7329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:37.663462 7329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:37.663507 7329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:37.663534 7329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:37.664891 7329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:37.665153 7329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:37.665481 7329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:37.670778 7335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.670950 7337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.670765 7334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.671087 7329 server_base.cc:1034] running on GCE node
I20260619 08:14:37.671340 7329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:37.671602 7329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:37.672768 7329 hybrid_clock.cc:648] HybridClock initialized: now 1781856877672756 us; error 52 us; skew 500 ppm
I20260619 08:14:37.674070 7329 webserver.cc:460] Webserver started at http://127.2.119.194:34321/ using document root <none> and password file <none>
I20260619 08:14:37.674321 7329 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.674392 7329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.674543 7329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.675616 7329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "7b4bb386de4b40fa8362f832a81e4dbc"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "54d83f21099423674dd37dd1df8664f8"
server_key_iv: "c39b6d5d532a7561ba028c5c674d22e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.675980 7329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "7b4bb386de4b40fa8362f832a81e4dbc"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "54d83f21099423674dd37dd1df8664f8"
server_key_iv: "c39b6d5d532a7561ba028c5c674d22e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.677266 7329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:37.678087 7343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.678272 7329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:37.678373 7329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "7b4bb386de4b40fa8362f832a81e4dbc"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "54d83f21099423674dd37dd1df8664f8"
server_key_iv: "c39b6d5d532a7561ba028c5c674d22e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.678455 7329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.681797 7329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.682051 7329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.682192 7329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.682415 7329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:37.682734 7329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:37.682811 7329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.682852 7329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:37.682879 7329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.688760 7329 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:33921
I20260619 08:14:37.688822 7456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:33921 every 8 connection(s)
I20260619 08:14:37.689109 7329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:37.692040 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.689801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:54895 (local address 127.2.119.254:45271)
0619 08:14:37.689844 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.689846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.690081 (+ 235us) server_negotiation.cc:407] Connection header received
0619 08:14:37.690533 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.690534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.690539 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.690602 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.691748 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.691749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.691749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.691754 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.691767 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.691771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.691820 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.691971 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.691984 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:14:37.692241 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.689760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45271 (local address 127.2.119.194:54895)
0619 08:14:37.689952 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.689994 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.690451 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.690687 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.690690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.690698 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.691603 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.691603 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.691631 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.691658 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.691872 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.691878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.691880 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.691958 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.692008 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260619 08:14:37.692780 7457 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45271
I20260619 08:14:37.692873 7457 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:37.693035 7457 heartbeater.cc:511] Master 127.2.119.254:45271 requested a full tablet report, sending...
I20260619 08:14:37.693434 7138 ts_manager.cc:194] Registered new tserver with Master: 7b4bb386de4b40fa8362f832a81e4dbc (127.2.119.194:33921)
I20260619 08:14:37.693567 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7329
I20260619 08:14:37.693670 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:37.693866 2527 external_mini_cluster.cc:1428] Setting key 7ef2150b23be094d67f957fbf5ac4ed2
I20260619 08:14:37.694653 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--builtin_ntp_servers=127.2.119.212:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 08:14:37.775485 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:37.775725 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:37.775760 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:37.776919 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:37.776989 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:37.777031 7460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:37.778371 7460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:37.778633 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:37.778960 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:37.784301 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.784377 7460 server_base.cc:1034] running on GCE node
W20260619 08:14:37.784301 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:37.784361 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:37.784795 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:37.784996 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:37.786144 7460 hybrid_clock.cc:648] HybridClock initialized: now 1781856877786135 us; error 31 us; skew 500 ppm
I20260619 08:14:37.787544 7460 webserver.cc:460] Webserver started at http://127.2.119.195:42787/ using document root <none> and password file <none>
I20260619 08:14:37.787761 7460 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:37.787802 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:37.787892 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:37.788972 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "555215185a9544d8986d9b9b69091334"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "19bbfce72d3dddd5b12b1f890e4d8b28"
server_key_iv: "2c3a30f757592d2eb1fbae6df1accd85"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.789283 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "555215185a9544d8986d9b9b69091334"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "19bbfce72d3dddd5b12b1f890e4d8b28"
server_key_iv: "2c3a30f757592d2eb1fbae6df1accd85"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.790632 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:37.791534 7474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.791718 7460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:37.791796 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "555215185a9544d8986d9b9b69091334"
format_stamp: "Formatted at 2026-06-19 08:14:37 on dist-test-slave-d0b5"
server_key: "19bbfce72d3dddd5b12b1f890e4d8b28"
server_key_iv: "2c3a30f757592d2eb1fbae6df1accd85"
server_key_version: "encryptionkey@0"
I20260619 08:14:37.791850 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:37.795210 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:37.795428 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:37.795521 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:37.795675 7460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:37.795917 7460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:37.795946 7460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.795966 7460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:37.795979 7460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:37.801728 7460 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44793
I20260619 08:14:37.801801 7587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44793 every 8 connection(s)
I20260619 08:14:37.802068 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:37.805109 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.802768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:53397 (local address 127.2.119.254:45271)
0619 08:14:37.802819 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.802822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.803050 (+ 228us) server_negotiation.cc:407] Connection header received
0619 08:14:37.803547 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.803548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.803554 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.803633 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.804825 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.804826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.804826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.804831 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.804845 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.804849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.804953 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.805041 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.805055 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:14:37.805277 7590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.802735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45271 (local address 127.2.119.195:53397)
0619 08:14:37.802915 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.802956 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.803417 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.803697 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.803701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.803709 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.804675 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.804675 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.804704 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.804731 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.804966 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.804970 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.804972 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.805031 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.805079 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260619 08:14:37.805850 7588 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45271
I20260619 08:14:37.805961 7588 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:37.806129 7588 heartbeater.cc:511] Master 127.2.119.254:45271 requested a full tablet report, sending...
I20260619 08:14:37.806492 7138 ts_manager.cc:194] Registered new tserver with Master: 555215185a9544d8986d9b9b69091334 (127.2.119.195:44793)
I20260619 08:14:37.809489 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7460
I20260619 08:14:37.809569 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:37.809724 2527 external_mini_cluster.cc:1428] Setting key 3391d6cd0717f7ff9b0135a32467a102
I20260619 08:14:37.810556 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:37.813053 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.811906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.2.119.254:45271)
0619 08:14:37.811955 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.811957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.812026 (+ 69us) server_negotiation.cc:407] Connection header received
0619 08:14:37.812085 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.812086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.812090 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.812129 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.812778 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.812779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.812779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.812782 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.812796 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.812800 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.812847 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.812878 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.812890 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:37.814832 7138 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:37.816457 7138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:37.820529 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.818677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41499 (local address 127.0.0.1:45326)
0619 08:14:37.819061 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.819071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.819119 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.819730 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.819731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.819734 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.820133 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.820134 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.820141 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.820154 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.820394 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.820395 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.820396 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.820469 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.820480 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260619 08:14:37.820600 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.818865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.2.119.193:41499)
0619 08:14:37.819365 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.819369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.819378 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:37.819403 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.819406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.819413 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.819476 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.820244 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.820252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.820252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.820256 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.820293 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.820324 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.820390 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.820495 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.820514 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":97,"threads_started":1}
I20260619 08:14:37.820725 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.819033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.2.119.195:44793)
0619 08:14:37.819228 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.819233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.819244 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:37.819270 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.819272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.819282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.819394 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.820210 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.820214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.820214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.820219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.820250 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.820264 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.820327 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.820603 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.820623 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260619 08:14:37.820528 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.818677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44793 (local address 127.0.0.1:50874)
0619 08:14:37.818824 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.818853 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.818930 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.819400 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.819406 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.819413 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.819938 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.819938 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.819956 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.819978 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.820335 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.820342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.820345 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.820433 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.820447 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260619 08:14:37.821022 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.819006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.2.119.194:33921)
0619 08:14:37.819251 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.819254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.819263 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:37.819285 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.819286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.819294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.819668 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.820551 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.820560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.820560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.820564 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.820596 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.820621 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.820676 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.820912 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.820925 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
I20260619 08:14:37.820849 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.818891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:33921 (local address 127.0.0.1:48522)
0619 08:14:37.819195 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.819204 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.819235 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.819968 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.819969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.819971 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.820447 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.820448 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.820455 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.820467 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.820704 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.820705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.820706 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.820748 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.820761 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":68,"threads_started":1}
I20260619 08:14:37.821718 7522 tablet_service.cc:1467] Processing CreateTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 (DEFAULT_TABLE table=test-table [id=90452194dd0549abb2b55fe59e9dee8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.822077 7522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1995f31b6c22482c8ba43d39e03ed826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.824482 7602 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Bootstrap starting.
I20260619 08:14:37.824823 7260 tablet_service.cc:1467] Processing CreateTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 (DEFAULT_TABLE table=test-table [id=90452194dd0549abb2b55fe59e9dee8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.825137 7260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1995f31b6c22482c8ba43d39e03ed826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.825205 7602 tablet_bootstrap.cc:654] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.825567 7602 log.cc:824] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.825753 7391 tablet_service.cc:1467] Processing CreateTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 (DEFAULT_TABLE table=test-table [id=90452194dd0549abb2b55fe59e9dee8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:37.826084 7391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1995f31b6c22482c8ba43d39e03ed826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:37.826385 7602 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: No bootstrap required, opened a new log
I20260619 08:14:37.826457 7602 ts_tablet_manager.cc:1397] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:37.828469 7604 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: Bootstrap starting.
I20260619 08:14:37.829078 7604 tablet_bootstrap.cc:654] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.829056 7602 raft_consensus.cc:348] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.829175 7602 raft_consensus.cc:374] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.829196 7602 raft_consensus.cc:729] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 555215185a9544d8986d9b9b69091334, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.829282 7602 consensus_queue.cc:260] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.829396 7604 log.cc:824] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.829615 7602 ts_tablet_manager.cc:1428] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:37.829643 7588 heartbeater.cc:503] Master 127.2.119.254:45271 was elected leader, sending a full tablet report...
I20260619 08:14:37.829771 7605 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Bootstrap starting.
I20260619 08:14:37.830405 7605 tablet_bootstrap.cc:654] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:37.830758 7605 log.cc:824] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:37.831651 7604 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: No bootstrap required, opened a new log
I20260619 08:14:37.831651 7605 tablet_bootstrap.cc:492] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: No bootstrap required, opened a new log
I20260619 08:14:37.831728 7604 ts_tablet_manager.cc:1397] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 08:14:37.831728 7605 ts_tablet_manager.cc:1397] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:37.834436 7605 raft_consensus.cc:348] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.834436 7604 raft_consensus.cc:348] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.834606 7605 raft_consensus.cc:374] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.834606 7604 raft_consensus.cc:374] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:37.834651 7604 raft_consensus.cc:729] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299cd8cfd3e04f6199460d9ea935be8d, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.834651 7605 raft_consensus.cc:729] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b4bb386de4b40fa8362f832a81e4dbc, State: Initialized, Role: FOLLOWER
I20260619 08:14:37.834807 7604 consensus_queue.cc:260] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.834800 7605 consensus_queue.cc:260] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.835075 7604 ts_tablet_manager.cc:1428] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 08:14:37.835075 7605 ts_tablet_manager.cc:1428] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:14:37.835125 7457 heartbeater.cc:503] Master 127.2.119.254:45271 was elected leader, sending a full tablet report...
I20260619 08:14:37.835191 7326 heartbeater.cc:503] Master 127.2.119.254:45271 was elected leader, sending a full tablet report...
I20260619 08:14:37.896832 7609 raft_consensus.cc:482] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:37.896970 7609 raft_consensus.cc:504] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.897326 7609 leader_election.cc:288] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 555215185a9544d8986d9b9b69091334 (127.2.119.195:44793), 299cd8cfd3e04f6199460d9ea935be8d (127.2.119.193:41499)
I20260619 08:14:37.898983 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.897526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44793 (local address 127.2.119.194:39881)
0619 08:14:37.897568 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.897581 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.897620 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.897885 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.897886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.897889 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.898534 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.898535 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.898541 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.898553 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.898776 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.898777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.898778 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.898862 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.898877 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:14:37.899106 7601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.897650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:39881 (local address 127.2.119.195:44793)
0619 08:14:37.897725 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.897727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.897738 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:37.897769 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.897770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.897775 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.897854 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.898645 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.898647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.898647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.898653 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.898671 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.898676 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.898737 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.899024 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.899039 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:14:37.899114 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.897527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41499 (local address 127.2.119.194:33385)
0619 08:14:37.897772 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.897806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:37.897865 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:37.898019 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:37.898020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:37.898032 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:37.898606 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:14:37.898607 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:37.898615 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:14:37.898630 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:37.898915 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:37.898916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:37.898917 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:14:37.899010 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 08:14:37.899024 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":50,"thread_start_us":111,"threads_started":1}
I20260619 08:14:37.899124 7599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.897709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:33385 (local address 127.2.119.193:41499)
0619 08:14:37.897770 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.897772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.897889 (+ 117us) server_negotiation.cc:407] Connection header received
0619 08:14:37.897935 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.897936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.897942 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.897997 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.898792 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.898793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.898794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.898799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.898818 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.898833 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.898892 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.899024 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.899039 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:37.899402 7542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1995f31b6c22482c8ba43d39e03ed826" candidate_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "555215185a9544d8986d9b9b69091334" is_pre_election: true
I20260619 08:14:37.899554 7542 raft_consensus.cc:2454] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b4bb386de4b40fa8362f832a81e4dbc in term 0.
I20260619 08:14:37.899533 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1995f31b6c22482c8ba43d39e03ed826" candidate_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "299cd8cfd3e04f6199460d9ea935be8d" is_pre_election: true
I20260619 08:14:37.899675 7280 raft_consensus.cc:2454] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b4bb386de4b40fa8362f832a81e4dbc in term 0.
I20260619 08:14:37.899869 7345 leader_election.cc:302] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555215185a9544d8986d9b9b69091334, 7b4bb386de4b40fa8362f832a81e4dbc; no voters:
I20260619 08:14:37.900000 7609 raft_consensus.cc:2781] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:37.900054 7609 raft_consensus.cc:482] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:37.900081 7609 raft_consensus.cc:3037] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.900812 7609 raft_consensus.cc:504] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.901013 7609 leader_election.cc:288] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [CANDIDATE]: Term 1 election: Requested vote from peers 555215185a9544d8986d9b9b69091334 (127.2.119.195:44793), 299cd8cfd3e04f6199460d9ea935be8d (127.2.119.193:41499)
I20260619 08:14:37.901156 7542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1995f31b6c22482c8ba43d39e03ed826" candidate_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "555215185a9544d8986d9b9b69091334"
I20260619 08:14:37.901185 7280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1995f31b6c22482c8ba43d39e03ed826" candidate_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "299cd8cfd3e04f6199460d9ea935be8d"
I20260619 08:14:37.901255 7280 raft_consensus.cc:3037] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.901263 7542 raft_consensus.cc:3037] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:37.902204 7542 raft_consensus.cc:2454] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b4bb386de4b40fa8362f832a81e4dbc in term 1.
I20260619 08:14:37.902205 7280 raft_consensus.cc:2454] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b4bb386de4b40fa8362f832a81e4dbc in term 1.
I20260619 08:14:37.902424 7345 leader_election.cc:302] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 555215185a9544d8986d9b9b69091334, 7b4bb386de4b40fa8362f832a81e4dbc; no voters:
I20260619 08:14:37.902534 7609 raft_consensus.cc:2781] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:37.902693 7609 raft_consensus.cc:686] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 1 LEADER]: Becoming Leader. State: Replica: 7b4bb386de4b40fa8362f832a81e4dbc, State: Running, Role: LEADER
I20260619 08:14:37.902818 7609 consensus_queue.cc:237] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } }
I20260619 08:14:37.903795 7137 catalog_manager.cc:5495] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b4bb386de4b40fa8362f832a81e4dbc (127.2.119.194). New cstate: current_term: 1 leader_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b4bb386de4b40fa8362f832a81e4dbc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33921 } health_report { overall_health: HEALTHY } } }
I20260619 08:14:37.914525 7600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.913560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.2.119.194:33921)
0619 08:14:37.913635 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:37.913637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:37.913647 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:37.913687 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:37.913688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:37.913700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:37.913756 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:37.914354 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:37.914354 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:37.914355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:37.914358 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:37.914370 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:37.914374 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:37.914423 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:37.914461 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 08:14:37.914471 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:14:37.916435 7280 raft_consensus.cc:1261] T 1995f31b6c22482c8ba43d39e03ed826 P 299cd8cfd3e04f6199460d9ea935be8d [term 1 FOLLOWER]: Refusing update from remote peer 7b4bb386de4b40fa8362f832a81e4dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:37.916484 7542 raft_consensus.cc:1261] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 1 FOLLOWER]: Refusing update from remote peer 7b4bb386de4b40fa8362f832a81e4dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:37.916815 7609 consensus_queue.cc:1035] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "299cd8cfd3e04f6199460d9ea935be8d" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:37.917018 7612 consensus_queue.cc:1035] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "555215185a9544d8986d9b9b69091334" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:37.919514 7615 mvcc.cc:205] Tried to move back new op lower bound from 7298485771942735872 to 7298485771891011584. Current Snapshot: MvccSnapshot[applied={T|T < 7298485771942735872}]
I20260619 08:14:37.919754 7617 mvcc.cc:205] Tried to move back new op lower bound from 7298485771942735872 to 7298485771891011584. Current Snapshot: MvccSnapshot[applied={T|T < 7298485771942735872}]
I20260619 08:14:37.922401 7138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:37.922505 7138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:37.923302 7138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3edba6047c1345899baff54d7923bff4: Sending DeleteTablet for 3 replicas of tablet 1995f31b6c22482c8ba43d39e03ed826
I20260619 08:14:37.923624 7522 tablet_service.cc:1514] Processing DeleteTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='slave'} at 127.0.0.1:50874
I20260619 08:14:37.923739 7260 tablet_service.cc:1514] Processing DeleteTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='slave'} at 127.0.0.1:45326
I20260619 08:14:37.923728 7391 tablet_service.cc:1514] Processing DeleteTablet for tablet 1995f31b6c22482c8ba43d39e03ed826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:37 UTC) from {username='slave'} at 127.0.0.1:48522
I20260619 08:14:37.923964 7626 tablet_replica.cc:330] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: stopping tablet replica
I20260619 08:14:37.924036 7624 tablet_replica.cc:330] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: stopping tablet replica
I20260619 08:14:37.924100 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7198
I20260619 08:14:37.924145 7624 raft_consensus.cc:2229] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:37.924137 7626 raft_consensus.cc:2229] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:37.924257 7624 raft_consensus.cc:2258] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:37.924396 7626 raft_consensus.cc:2258] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:37.925465 7626 ts_tablet_manager.cc:1905] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:37.927423 7626 ts_tablet_manager.cc:1918] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:37.927503 7626 log.cc:1196] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/1995f31b6c22482c8ba43d39e03ed826
I20260619 08:14:37.927640 7624 ts_tablet_manager.cc:1905] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:37.928049 7626 ts_tablet_manager.cc:1939] T 1995f31b6c22482c8ba43d39e03ed826 P 7b4bb386de4b40fa8362f832a81e4dbc: Deleting consensus metadata
I20260619 08:14:37.928454 7124 catalog_manager.cc:4841] TS 7b4bb386de4b40fa8362f832a81e4dbc (127.2.119.194:33921): tablet 1995f31b6c22482c8ba43d39e03ed826 (table test-table [id=90452194dd0549abb2b55fe59e9dee8b]) successfully deleted
I20260619 08:14:37.929494 7624 ts_tablet_manager.cc:1918] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 08:14:37.929558 7124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:41499 (error 108)
I20260619 08:14:37.929677 7624 log.cc:1196] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/1995f31b6c22482c8ba43d39e03ed826
I20260619 08:14:37.929811 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7329
I20260619 08:14:37.930012 7624 ts_tablet_manager.cc:1939] T 1995f31b6c22482c8ba43d39e03ed826 P 555215185a9544d8986d9b9b69091334: Deleting consensus metadata
I20260619 08:14:37.931095 7123 catalog_manager.cc:4841] TS 555215185a9544d8986d9b9b69091334 (127.2.119.195:44793): tablet 1995f31b6c22482c8ba43d39e03ed826 (table test-table [id=90452194dd0549abb2b55fe59e9dee8b]) successfully deleted
I20260619 08:14:37.931581 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:37.931437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41499 (local address 127.0.0.1:45342)
0619 08:14:37.931500 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:37.931541 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:41499: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 08:14:37.931761 7124 catalog_manager.cc:4584] TS 299cd8cfd3e04f6199460d9ea935be8d (127.2.119.193:41499): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1995f31b6c22482c8ba43d39e03ed826: Network error: Client connection negotiation failed: client connection to 127.2.119.193:41499: connect: Connection refused (error 111)
I20260619 08:14:37.935187 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7460
I20260619 08:14:37.942348 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7107
2026-06-19T08:14:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (675 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 08:14:37.961895 2527 mini_oidc.cc:152] Starting JWKS server
I20260619 08:14:37.963014 2527 webserver.cc:460] Webserver started at https://127.0.0.1:41873/ using document root <none> and password file <none>
I20260619 08:14:37.963141 2527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:14:37.963326 2527 webserver.cc:460] Webserver started at http://127.0.0.1:36305/ using document root <none> and password file <none>
2026-06-19T08:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:37Z Disabled control of system clock
I20260619 08:14:37.971235 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:46479
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:46479
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41873/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/testchainca.cert with env {}
W20260619 08:14:38.051916 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:38.052126 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:38.052148 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:38.053246 7633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:38.053280 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:38.053309 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:38.053323 7633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:38.053335 7633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:14:38.053347 7633 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:14:38.053359 7633 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41873/jwks.json
I20260619 08:14:38.054978 7633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:46479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41873/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:38.055240 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:38.055411 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:38.060315 7638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.060508 7639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.060544 7641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:38.060596 7633 server_base.cc:1034] running on GCE node
I20260619 08:14:38.060878 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:38.061193 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:38.062361 7633 hybrid_clock.cc:648] HybridClock initialized: now 1781856878062333 us; error 40 us; skew 500 ppm
I20260619 08:14:38.063524 7633 webserver.cc:460] Webserver started at http://127.2.119.254:35089/ using document root <none> and password file <none>
I20260619 08:14:38.063716 7633 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:38.063757 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:38.063844 7633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:38.064941 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "487fb62a033648199404c61fbde9f194"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "315d859b57531dd7e2b42153bc5e4731"
server_key_iv: "f84210de93be410a8fd9049639b43b73"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.065371 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "487fb62a033648199404c61fbde9f194"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "315d859b57531dd7e2b42153bc5e4731"
server_key_iv: "f84210de93be410a8fd9049639b43b73"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.066596 7633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:38.067265 7647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:38.067449 7633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:38.067581 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "487fb62a033648199404c61fbde9f194"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "315d859b57531dd7e2b42153bc5e4731"
server_key_iv: "f84210de93be410a8fd9049639b43b73"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.067669 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:38.083808 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:38.084188 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:38.084352 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:38.088603 7633 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:46479
I20260619 08:14:38.088640 7699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:46479 every 8 connection(s)
I20260619 08:14:38.089010 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:38.089990 7700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:38.092284 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194: Bootstrap starting.
I20260619 08:14:38.092916 7700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:38.093209 7700 log.cc:824] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:38.093906 7700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194: No bootstrap required, opened a new log
I20260619 08:14:38.095490 7700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } }
I20260619 08:14:38.095655 7700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:38.095702 7700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 487fb62a033648199404c61fbde9f194, State: Initialized, Role: FOLLOWER
I20260619 08:14:38.095791 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7633
I20260619 08:14:38.095844 7700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } }
I20260619 08:14:38.095929 7700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:38.095963 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:38.095974 7700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:38.096016 7700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:38.096171 2527 external_mini_cluster.cc:1428] Setting key 1b77afb17d7937fdc89e0b7996746d1b
I20260619 08:14:38.096812 7700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } }
I20260619 08:14:38.096926 7700 leader_election.cc:302] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 487fb62a033648199404c61fbde9f194; no voters:
I20260619 08:14:38.097075 7700 leader_election.cc:288] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:38.097137 7705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:38.097350 7700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:38.097335 7705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [term 1 LEADER]: Becoming Leader. State: Replica: 487fb62a033648199404c61fbde9f194, State: Running, Role: LEADER
I20260619 08:14:38.097491 7705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } }
I20260619 08:14:38.097802 7707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 487fb62a033648199404c61fbde9f194. Latest consensus state: current_term: 1 leader_uuid: "487fb62a033648199404c61fbde9f194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } } }
I20260619 08:14:38.097934 7707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.097802 7706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "487fb62a033648199404c61fbde9f194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "487fb62a033648199404c61fbde9f194" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 46479 } } }
I20260619 08:14:38.098024 7706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.098686 7712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:38.099198 7712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:38.100816 7712 catalog_manager.cc:1347] Generated new cluster ID: 4b8bee97c10a4c04b37d9df0af92f6dd
I20260619 08:14:38.100871 7712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:38.102875 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.096676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51308 (local address 127.2.119.254:46479)
0619 08:14:38.096923 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.096932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.096966 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:14:38.097088 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.097096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.097174 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.097628 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:38.097719 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.099619 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.100664 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.101327 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.101439 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.102169 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:38.102172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:38.102173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:38.102367 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:38.102406 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:38.102420 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:38.102490 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:38.102595 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:14:38.102669 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260619 08:14:38.103865 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:14:38.121565 7712 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:38.122146 7712 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:38.127337 7712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 487fb62a033648199404c61fbde9f194: Generated new TSK 0
I20260619 08:14:38.127556 7712 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:14:38.150723 7730 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 08:14:38.150832 7730 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:14:38.151108 7704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.134983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51310 (local address 127.2.119.254:46479)
0619 08:14:38.135033 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.135036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.135138 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:14:38.135243 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.135244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.135254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.135319 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:14:38.135516 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.135993 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.137178 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.137535 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.137593 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.137830 (+ 237us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:14:38.150853 (+ 13023us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:14:38.151036 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 08:14:38.151172 7704 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 08:14:38.151458 7729 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:38.134883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:46479 (local address 127.0.0.1:51310)
0619 08:14:38.135059 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:38.135070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:38.135137 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:38.135352 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:38.135353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:38.135355 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:14:38.135437 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.135440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.136078 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.136079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.137067 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.137070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.137613 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.137614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.137686 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.137711 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:14:38.151026 (+ 13315us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 08:14:38.151140 (+ 114us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.119.254:46479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.119.254:46479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41873/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 08:14:38.151893 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7633
2026-06-19T08:14:38Z chronyd exiting
I20260619 08:14:38.187956 2527 test_util.cc:175] -----------------------------------------------
I20260619 08:14:38.188045 2527 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781856856097588-2527-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 08:14:38.188848 2527 mini_oidc.cc:152] Starting JWKS server
I20260619 08:14:38.189677 2527 webserver.cc:460] Webserver started at https://127.0.0.1:34693/ using document root <none> and password file <none>
I20260619 08:14:38.189755 2527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:14:38.189908 2527 webserver.cc:460] Webserver started at http://127.0.0.1:41521/ using document root <none> and password file <none>
2026-06-19T08:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:38Z Disabled control of system clock
I20260619 08:14:38.197834 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36979
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:45217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36979
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34693/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/testchainca.cert with env {}
W20260619 08:14:38.278461 7736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:38.278673 7736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:38.278697 7736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:38.279877 7736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:38.279924 7736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:38.279937 7736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:38.279950 7736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:38.279963 7736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:14:38.279975 7736 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:14:38.280022 7736 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34693/jwks.json
I20260619 08:14:38.281598 7736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34693/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:38.281872 7736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:38.282104 7736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:38.287108 7742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.287094 7744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:38.287220 7736 server_base.cc:1034] running on GCE node
W20260619 08:14:38.287089 7741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:38.287601 7736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:38.287890 7736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:38.289058 7736 hybrid_clock.cc:648] HybridClock initialized: now 1781856878289060 us; error 42 us; skew 500 ppm
I20260619 08:14:38.290263 7736 webserver.cc:460] Webserver started at http://127.2.119.254:35267/ using document root <none> and password file <none>
I20260619 08:14:38.290518 7736 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:38.290592 7736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:38.290737 7736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:38.291877 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "947c4288b7e7421997a9e4dea71e57a9"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "7aa8b0511fc42b8a1461cc7cdfc2fd25"
server_key_iv: "1b0e60526b9ebce465e77c3831dd6f18"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.292227 7736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "947c4288b7e7421997a9e4dea71e57a9"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "7aa8b0511fc42b8a1461cc7cdfc2fd25"
server_key_iv: "1b0e60526b9ebce465e77c3831dd6f18"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.293615 7736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:38.294334 7750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:38.294518 7736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:38.294620 7736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "947c4288b7e7421997a9e4dea71e57a9"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "7aa8b0511fc42b8a1461cc7cdfc2fd25"
server_key_iv: "1b0e60526b9ebce465e77c3831dd6f18"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.294692 7736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:38.331094 7736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:38.331440 7736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:38.331604 7736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:38.335441 7736 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36979
I20260619 08:14:38.335520 7802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36979 every 8 connection(s)
I20260619 08:14:38.335865 7736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:38.336942 7803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:38.339286 7803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9: Bootstrap starting.
I20260619 08:14:38.339898 7803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:38.340178 7803 log.cc:824] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:38.340958 7803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9: No bootstrap required, opened a new log
I20260619 08:14:38.342463 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7736
I20260619 08:14:38.342574 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:38.342765 2527 external_mini_cluster.cc:1428] Setting key 50829a7b35ee01a03e4be656f5e8d70f
I20260619 08:14:38.342983 7803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } }
I20260619 08:14:38.343155 7803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:38.343199 7803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947c4288b7e7421997a9e4dea71e57a9, State: Initialized, Role: FOLLOWER
I20260619 08:14:38.343350 7803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } }
I20260619 08:14:38.343451 7803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:38.343490 7803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:38.343554 7803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:38.344273 7803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } }
I20260619 08:14:38.344364 7803 leader_election.cc:302] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 947c4288b7e7421997a9e4dea71e57a9; no voters:
I20260619 08:14:38.344579 7803 leader_election.cc:288] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:38.344754 7808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:38.344862 7808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [term 1 LEADER]: Becoming Leader. State: Replica: 947c4288b7e7421997a9e4dea71e57a9, State: Running, Role: LEADER
I20260619 08:14:38.344875 7803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:38.344959 7808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } }
I20260619 08:14:38.345311 7808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947c4288b7e7421997a9e4dea71e57a9. Latest consensus state: current_term: 1 leader_uuid: "947c4288b7e7421997a9e4dea71e57a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } } }
I20260619 08:14:38.345393 7808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.345562 7809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "947c4288b7e7421997a9e4dea71e57a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947c4288b7e7421997a9e4dea71e57a9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36979 } } }
I20260619 08:14:38.345635 7809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.345763 7813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:38.346540 7813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:38.348270 7813 catalog_manager.cc:1347] Generated new cluster ID: 0601d04555c54e3096fd011b6e325dcf
I20260619 08:14:38.348333 7813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:38.348922 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.343305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.2.119.254:36979)
0619 08:14:38.343560 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.343570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.343604 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:14:38.343714 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.343719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.343770 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.343875 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:38.344113 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.345396 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.346739 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.347323 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.347480 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.348177 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:38.348181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:38.348181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:38.348404 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:38.348448 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:38.348463 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:38.348538 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:38.348626 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:14:38.348705 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":130,"threads_started":1}
I20260619 08:14:38.349854 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:14:38.366917 7813 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:38.367681 7813 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:38.370869 7813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 947c4288b7e7421997a9e4dea71e57a9: Generated new TSK 0
I20260619 08:14:38.371047 7813 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 08:14:38.390326 7833 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 08:14:38.390409 7833 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 08:14:38.390733 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.376265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54320 (local address 127.2.119.254:36979)
0619 08:14:38.376309 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.376311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.376413 (+ 102us) server_negotiation.cc:407] Connection header received
0619 08:14:38.376506 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.376507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.376520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.376604 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:14:38.376775 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.377606 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.378369 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.378698 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.378768 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.378938 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:14:38.390502 (+ 11564us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:14:38.390664 (+ 162us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54320: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 08:14:38.390786 7807 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54320: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 08:14:38.390810 7832 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:38.376175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36979 (local address 127.0.0.1:54320)
0619 08:14:38.376314 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:38.376324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:38.376401 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:38.376634 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:38.376636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:38.376638 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:14:38.376693 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.376695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.377695 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.377695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.378279 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.378281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.378778 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.378779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.378834 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.378839 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:14:38.390600 (+ 11761us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 08:14:38.390719 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.119.254:36979: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34693/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":93,"threads_started":1}
I20260619 08:14:38.391135 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7736
2026-06-19T08:14:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 08:14:38.428730 2527 mini_oidc.cc:152] Starting JWKS server
I20260619 08:14:38.429569 2527 webserver.cc:460] Webserver started at https://127.0.0.1:37055/ using document root <none> and password file <none>
I20260619 08:14:38.429647 2527 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 08:14:38.429808 2527 webserver.cc:460] Webserver started at http://127.0.0.1:36943/ using document root <none> and password file <none>
2026-06-19T08:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:38Z Disabled control of system clock
I20260619 08:14:38.437851 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40713
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:40713
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37055/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:14:38.519433 7839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:38.519642 7839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:38.519665 7839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:38.520874 7839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:38.520924 7839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:38.520937 7839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:38.520951 7839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:38.520962 7839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:14:38.520975 7839 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:14:38.520987 7839 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37055/jwks.json
I20260619 08:14:38.522552 7839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:40713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37055/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:38.522835 7839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:38.523149 7839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:38.528496 7845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.528533 7844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.528754 7847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:38.528823 7839 server_base.cc:1034] running on GCE node
I20260619 08:14:38.529141 7839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:38.529556 7839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:38.530937 7839 hybrid_clock.cc:648] HybridClock initialized: now 1781856878530923 us; error 32 us; skew 500 ppm
I20260619 08:14:38.532621 7839 webserver.cc:460] Webserver started at http://127.2.119.254:44361/ using document root <none> and password file <none>
I20260619 08:14:38.532917 7839 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:38.532970 7839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:38.533162 7839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:38.534693 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "934d0ad8c7b282de35d8db3b86b9cf03"
server_key_iv: "74d1c9265a339ce0f8f7f05376aec4e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.535179 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "934d0ad8c7b282de35d8db3b86b9cf03"
server_key_iv: "74d1c9265a339ce0f8f7f05376aec4e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.536638 7839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:38.537573 7853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:38.537875 7839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:38.538008 7839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "934d0ad8c7b282de35d8db3b86b9cf03"
server_key_iv: "74d1c9265a339ce0f8f7f05376aec4e9"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.538106 7839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:38.553191 7839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:38.553591 7839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:38.553754 7839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:38.558145 7839 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:40713
I20260619 08:14:38.558193 7905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:40713 every 8 connection(s)
I20260619 08:14:38.558548 7839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:38.559235 7906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:38.561873 7906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0: Bootstrap starting.
I20260619 08:14:38.562143 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7839
I20260619 08:14:38.562255 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:38.562458 2527 external_mini_cluster.cc:1428] Setting key b96720f2ed98a8f41ff2f111ac93e529
I20260619 08:14:38.562513 7906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:38.562899 7906 log.cc:824] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:38.563690 7906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0: No bootstrap required, opened a new log
I20260619 08:14:38.565589 7906 raft_consensus.cc:348] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } }
I20260619 08:14:38.565758 7906 raft_consensus.cc:374] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:38.565900 7906 raft_consensus.cc:729] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55aa5dc5a3f4bebaa4d15a2ec4f6df0, State: Initialized, Role: FOLLOWER
I20260619 08:14:38.566051 7906 consensus_queue.cc:260] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } }
I20260619 08:14:38.566139 7906 raft_consensus.cc:388] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:38.566184 7906 raft_consensus.cc:482] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:38.566236 7906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:38.566803 7906 raft_consensus.cc:504] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } }
I20260619 08:14:38.566936 7906 leader_election.cc:302] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c55aa5dc5a3f4bebaa4d15a2ec4f6df0; no voters:
I20260619 08:14:38.567104 7906 leader_election.cc:288] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:38.567157 7911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:38.567302 7911 raft_consensus.cc:686] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [term 1 LEADER]: Becoming Leader. State: Replica: c55aa5dc5a3f4bebaa4d15a2ec4f6df0, State: Running, Role: LEADER
I20260619 08:14:38.567373 7906 sys_catalog.cc:564] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:38.567405 7911 consensus_queue.cc:237] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } }
I20260619 08:14:38.567736 7911 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c55aa5dc5a3f4bebaa4d15a2ec4f6df0. Latest consensus state: current_term: 1 leader_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } } }
I20260619 08:14:38.567878 7911 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.567747 7912 sys_catalog.cc:455] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c55aa5dc5a3f4bebaa4d15a2ec4f6df0" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40713 } } }
I20260619 08:14:38.568169 7918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:38.568193 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.563026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.2.119.254:40713)
0619 08:14:38.563252 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.563262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.563309 (+ 47us) server_negotiation.cc:407] Connection header received
0619 08:14:38.563765 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.563771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.563872 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.564027 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:38.564200 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.565078 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.565989 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.566514 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.566663 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.567416 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:38.567420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:38.567420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:38.567621 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:38.567668 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:38.567684 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:38.567726 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:38.567866 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 08:14:38.567954 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260619 08:14:38.568027 7912 sys_catalog.cc:458] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.569130 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:14:38.569387 7918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:38.571010 7918 catalog_manager.cc:1347] Generated new cluster ID: 1fa2ac0854334e0a8fa83bb0a726c570
I20260619 08:14:38.571064 7918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:38.578552 7918 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:38.579082 7918 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:38.582305 7918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c55aa5dc5a3f4bebaa4d15a2ec4f6df0: Generated new TSK 0
I20260619 08:14:38.582495 7918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:14:38.596728 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.592769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.2.119.254:40713)
0619 08:14:38.592818 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.592820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.592857 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:14:38.592945 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.592945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.592963 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.593031 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:14:38.593183 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.593784 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.595070 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.595284 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.595346 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.595535 (+ 189us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:14:38.596449 (+ 914us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37055/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:14:38.596631 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59626: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37055/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:14:38.596804 7935 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:38.592591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:40713 (local address 127.0.0.1:59626)
0619 08:14:38.592796 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:38.592807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:38.592874 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:38.593048 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:38.593049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:38.593051 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:14:38.593112 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.593115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.593888 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.593889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.594974 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.594977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.595367 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.595368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.595430 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.595437 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:14:38.596584 (+ 1147us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37055/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 08:14:38.596685 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.119.254:40713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37055/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":128,"threads_started":1}
W20260619 08:14:38.596807 7910 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59626: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37055/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260619 08:14:38.597129 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7839
E20260619 08:14:38.607918 7936 webserver.cc:533] Webserver: sslize failed:
E20260619 08:14:38.607996 7936 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T08:14:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (206 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T08:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:38Z Disabled control of system clock
I20260619 08:14:38.642721 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43335
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:43335
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/testchainca.cert with env {}
W20260619 08:14:38.723277 7940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:38.723503 7940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:38.723527 7940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:38.724735 7940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:38.724781 7940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:38.724793 7940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:38.724807 7940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:38.724818 7940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 08:14:38.724881 7940 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 08:14:38.724900 7940 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 08:14:38.726506 7940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:43335
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:38.726779 7940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:38.727006 7940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:14:38.732270 7940 server_base.cc:1034] running on GCE node
W20260619 08:14:38.732240 7946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.732240 7948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:38.732240 7945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:38.732693 7940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:38.732957 7940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:38.734127 7940 hybrid_clock.cc:648] HybridClock initialized: now 1781856878734104 us; error 42 us; skew 500 ppm
I20260619 08:14:38.735733 7940 webserver.cc:460] Webserver started at http://127.2.119.254:37599/ using document root <none> and password file <none>
I20260619 08:14:38.736013 7940 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:38.736064 7940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:38.736255 7940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:38.737593 7940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "a8135268447f46aabc0e969fe19061f6"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "d541f56364650aa5bbf9d678d2292ac1"
server_key_iv: "536e017fdbc030f94d9bd7c3e36cadb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.738001 7940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "a8135268447f46aabc0e969fe19061f6"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "d541f56364650aa5bbf9d678d2292ac1"
server_key_iv: "536e017fdbc030f94d9bd7c3e36cadb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.739497 7940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:38.740389 7954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:38.740615 7940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:38.740746 7940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "a8135268447f46aabc0e969fe19061f6"
format_stamp: "Formatted at 2026-06-19 08:14:38 on dist-test-slave-d0b5"
server_key: "d541f56364650aa5bbf9d678d2292ac1"
server_key_iv: "536e017fdbc030f94d9bd7c3e36cadb0"
server_key_version: "encryptionkey@0"
I20260619 08:14:38.740835 7940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:38.759601 7940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:38.759974 7940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:38.760142 7940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:38.764122 7940 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:43335
I20260619 08:14:38.764155 8006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:43335 every 8 connection(s)
I20260619 08:14:38.764511 7940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:38.765573 8007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:38.767179 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 7940
I20260619 08:14:38.767282 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:38.767465 2527 external_mini_cluster.cc:1428] Setting key ff6bdf494e4f208f91d3fc52f80300eb
I20260619 08:14:38.768347 8007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6: Bootstrap starting.
I20260619 08:14:38.769033 8007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:38.769392 8007 log.cc:824] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:38.770201 8007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6: No bootstrap required, opened a new log
I20260619 08:14:38.772439 8007 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } }
I20260619 08:14:38.772617 8007 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:38.772655 8007 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8135268447f46aabc0e969fe19061f6, State: Initialized, Role: FOLLOWER
I20260619 08:14:38.772789 8007 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } }
I20260619 08:14:38.772869 8007 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:38.772893 8007 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:38.772960 8007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:38.773705 8007 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } }
I20260619 08:14:38.773820 8007 leader_election.cc:302] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8135268447f46aabc0e969fe19061f6; no voters:
I20260619 08:14:38.774029 8012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:38.774003 8007 leader_election.cc:288] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:38.774207 8012 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [term 1 LEADER]: Becoming Leader. State: Replica: a8135268447f46aabc0e969fe19061f6, State: Running, Role: LEADER
I20260619 08:14:38.774246 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.767977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.2.119.254:43335)
0619 08:14:38.768199 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.768208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.768241 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:14:38.768804 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.768813 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.768900 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.769067 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:38.769389 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.770743 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.771752 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.772459 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.772586 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.773434 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:38.773439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:38.773440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:14:38.773649 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:38.773695 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:14:38.773710 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:38.773784 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:38.773898 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:14:38.773993 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260619 08:14:38.774298 8012 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } }
I20260619 08:14:38.774381 8007 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:38.774672 8012 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8135268447f46aabc0e969fe19061f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } } }
I20260619 08:14:38.774784 8012 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.775009 8017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:38.775285 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:14:38.774678 8013 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8135268447f46aabc0e969fe19061f6. Latest consensus state: current_term: 1 leader_uuid: "a8135268447f46aabc0e969fe19061f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8135268447f46aabc0e969fe19061f6" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43335 } } }
I20260619 08:14:38.775722 8017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:38.775797 8013 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:38.778143 8017 catalog_manager.cc:1347] Generated new cluster ID: 26f5cae602b0460bb52e35905027a9c0
I20260619 08:14:38.778223 8017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:38.813634 8017 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:38.814266 8017 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:38.819485 8017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8135268447f46aabc0e969fe19061f6: Generated new TSK 0
I20260619 08:14:38.819720 8017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:14:38.870299 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:38.832456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.2.119.254:43335)
0619 08:14:38.832532 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:38.832534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:38.832652 (+ 118us) server_negotiation.cc:407] Connection header received
0619 08:14:38.832760 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:38.832760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:38.832779 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:38.832867 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0619 08:14:38.833035 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.833632 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.834478 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.834674 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.834751 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.834985 (+ 234us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 08:14:38.869985 (+ 35000us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:14:38.870211 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59396: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 08:14:38.870386 8010 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59396: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 08:14:38.870386 8036 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:38.832388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43335 (local address 127.0.0.1:59396)
0619 08:14:38.832576 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:38.832588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:38.832671 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:38.832885 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:38.832886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:38.832888 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 08:14:38.832960 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.832962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.833738 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.833740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.834371 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:38.834373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:38.834763 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:38.834780 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:38.834841 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:38.834853 (+ 12us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 08:14:38.870146 (+ 35293us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 08:14:38.870278 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.119.254:43335: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260619 08:14:38.870709 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 7940
2026-06-19T08:14:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (252 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:39 dist-test-slave-d0b5 krb5kdc[8048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:39 dist-test-slave-d0b5 krb5kdc[8048](info): set up 2 sockets
Jun 19 08:14:39 dist-test-slave-d0b5 krb5kdc[8048](info): commencing operation
krb5kdc: starting...
W20260619 08:14:40.106498 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:40.229123 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36703
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:40.309255 8061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:40.309561 8061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:40.309600 8061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:40.310775 8061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:40.310854 8061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:40.310878 8061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:40.310900 8061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:40.310928 8061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:40.312364 8061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:40.312635 8061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:40.312867 8061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:40.317790 8069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.317781 8066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.317917 8067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:40.317929 8061 server_base.cc:1034] running on GCE node
I20260619 08:14:40.318253 8061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:40.318554 8061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:40.319723 8061 hybrid_clock.cc:648] HybridClock initialized: now 1781856880319719 us; error 37 us; skew 500 ppm
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.322427 8061 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:40.322949 8061 webserver.cc:460] Webserver started at http://127.2.119.254:44653/ using document root <none> and password file <none>
I20260619 08:14:40.323191 8061 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:40.323263 8061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:40.323408 8061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:40.324390 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "057fbead1ea14ec1b442b4b3ee7f5c22"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "9c2d9987a342b7de77b760b731c74110"
server_key_iv: "846d408492e7b5fb223bdfe4026f7b5c"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.324702 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "057fbead1ea14ec1b442b4b3ee7f5c22"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "9c2d9987a342b7de77b760b731c74110"
server_key_iv: "846d408492e7b5fb223bdfe4026f7b5c"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.326107 8061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:40.326848 8076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.327054 8061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:40.327138 8061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "057fbead1ea14ec1b442b4b3ee7f5c22"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "9c2d9987a342b7de77b760b731c74110"
server_key_iv: "846d408492e7b5fb223bdfe4026f7b5c"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.327193 8061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:40.353611 8061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:40.355947 8061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:40.356141 8061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:40.360337 8061 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36703
I20260619 08:14:40.360440 8128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36703 every 8 connection(s)
I20260619 08:14:40.360749 8061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:40.361667 8129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:40.363968 8129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22: Bootstrap starting.
I20260619 08:14:40.364045 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8061
I20260619 08:14:40.364132 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:40.364331 2527 external_mini_cluster.cc:1428] Setting key b607b3ad89689df45d9d4a9d1bed6b3a
I20260619 08:14:40.364671 8129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:40.365264 8129 log.cc:824] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:40.366061 8129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22: No bootstrap required, opened a new log
I20260619 08:14:40.368237 8129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } }
I20260619 08:14:40.368402 8129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:40.368435 8129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057fbead1ea14ec1b442b4b3ee7f5c22, State: Initialized, Role: FOLLOWER
I20260619 08:14:40.368594 8129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } }
I20260619 08:14:40.368674 8129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:40.368697 8129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:40.368772 8129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:40.369418 8129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } }
I20260619 08:14:40.369549 8129 leader_election.cc:302] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 057fbead1ea14ec1b442b4b3ee7f5c22; no voters:
I20260619 08:14:40.369735 8129 leader_election.cc:288] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:40.369779 8134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:40.369951 8134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [term 1 LEADER]: Becoming Leader. State: Replica: 057fbead1ea14ec1b442b4b3ee7f5c22, State: Running, Role: LEADER
I20260619 08:14:40.369988 8129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:40.370051 8134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } }
I20260619 08:14:40.370369 8135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } } }
I20260619 08:14:40.370456 8135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:40.370353 8134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057fbead1ea14ec1b442b4b3ee7f5c22. Latest consensus state: current_term: 1 leader_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "057fbead1ea14ec1b442b4b3ee7f5c22" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36703 } } }
I20260619 08:14:40.370671 8134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:40.370796 8140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:40.371516 8140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:40.373215 8140 catalog_manager.cc:1347] Generated new cluster ID: 1729c780cf7a49539ae292919c5b8f53
I20260619 08:14:40.373265 8140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:40.375622 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.364864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.2.119.254:36703)
0619 08:14:40.365053 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.365061 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.365086 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:14:40.365617 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.365624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.365671 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.365794 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.366557 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.367444 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.368558 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.369132 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.369214 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.372220 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.372225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.372225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.372238 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.374096 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.374477 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.374480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.374481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.374571 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.374816 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.374817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.374817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.375055 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.375220 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.375315 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.375458 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260619 08:14:40.384801 8140 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:40.385538 8140 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:40.389110 8140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 057fbead1ea14ec1b442b4b3ee7f5c22: Generated new TSK 0
I20260619 08:14:40.389274 8140 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:40.417421 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:40.499823 8155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:40.500034 8155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:40.500058 8155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:40.501240 8155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:40.501283 8155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:40.501328 8155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:40.502869 8155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:40.503130 8155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:40.503378 8155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:40.508464 8160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.508514 8163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.508514 8161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:40.508709 8155 server_base.cc:1034] running on GCE node
I20260619 08:14:40.508913 8155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:40.509188 8155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:40.510331 8155 hybrid_clock.cc:648] HybridClock initialized: now 1781856880510308 us; error 42 us; skew 500 ppm
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.512831 8155 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:40.513360 8155 webserver.cc:460] Webserver started at http://127.2.119.193:44279/ using document root <none> and password file <none>
I20260619 08:14:40.513588 8155 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:40.513634 8155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:40.513765 8155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:40.514859 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "32d1392bba084cb9b1458991f1217dc3"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "d01b3e6804a8f37be5cac1b358a3f277"
server_key_iv: "8bb845ad72b84cc2aec51399d1094848"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.515232 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "32d1392bba084cb9b1458991f1217dc3"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "d01b3e6804a8f37be5cac1b358a3f277"
server_key_iv: "8bb845ad72b84cc2aec51399d1094848"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.516580 8155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:40.517431 8170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.517614 8155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:40.517714 8155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "32d1392bba084cb9b1458991f1217dc3"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "d01b3e6804a8f37be5cac1b358a3f277"
server_key_iv: "8bb845ad72b84cc2aec51399d1094848"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.517798 8155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:40.532541 8155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:40.535149 8155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:40.535362 8155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:40.535641 8155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:40.536021 8155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:40.536088 8155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.536125 8155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:40.536157 8155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.542459 8155 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:36959
I20260619 08:14:40.542517 8283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:36959 every 8 connection(s)
I20260619 08:14:40.542899 8155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:40.552523 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8155
I20260619 08:14:40.552675 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:40.552728 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.543723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33235 (local address 127.2.119.254:36703)
0619 08:14:40.543773 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.543775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.544025 (+ 250us) server_negotiation.cc:407] Connection header received
0619 08:14:40.545011 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.545012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.545024 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.545079 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.545799 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.546328 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.547386 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.547600 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.547686 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.549996 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.549997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.549998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.550007 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.551272 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.551733 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.551735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.551735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.551794 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.552058 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.552059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.552059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.552171 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.552246 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.552546 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.552638 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:14:40.552928 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.543692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36703 (local address 127.2.119.193:33235)
0619 08:14:40.543883 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:40.543933 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:40.544882 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:40.545158 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:40.545163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:40.545505 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:40.545697 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.545701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.546455 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.546456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.547279 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.547281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.547691 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.547692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.547840 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.548391 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:40.548399 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:40.549891 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:40.551391 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.551398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.551400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.551636 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.551874 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.551875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.551875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.551980 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.552247 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:40.552254 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:40.552408 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 08:14:40.552513 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 08:14:40.552668 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260619 08:14:40.553009 2527 external_mini_cluster.cc:1428] Setting key fa3114422e82d951cfe0eb997289d85d
I20260619 08:14:40.553580 8284 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36703
I20260619 08:14:40.553679 8284 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:40.553877 8284 heartbeater.cc:511] Master 127.2.119.254:36703 requested a full tablet report, sending...
I20260619 08:14:40.554438 8092 ts_manager.cc:194] Registered new tserver with Master: 32d1392bba084cb9b1458991f1217dc3 (127.2.119.193:36959)
I20260619 08:14:40.555090 8092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:33235
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:40.580937 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:40.660195 8289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:40.660398 8289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:40.660421 8289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:40.661733 8289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:40.661777 8289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:40.661803 8289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:40.663259 8289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:40.663522 8289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:40.663754 8289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:40.669021 8294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.669116 8297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.669064 8295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:40.669143 8289 server_base.cc:1034] running on GCE node
I20260619 08:14:40.669416 8289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:40.669693 8289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:40.670845 8289 hybrid_clock.cc:648] HybridClock initialized: now 1781856880670817 us; error 48 us; skew 500 ppm
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.673277 8289 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:40.673765 8289 webserver.cc:460] Webserver started at http://127.2.119.194:39041/ using document root <none> and password file <none>
I20260619 08:14:40.674016 8289 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:40.674083 8289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:40.674208 8289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:40.675303 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "5e551d3e46e24c7aa47605aefaa91568"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "74ace23a73ff3f4f37f9ed4a880fea8e"
server_key_iv: "c27d9f88a26641f5b664a8b36d94749e"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.675668 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "5e551d3e46e24c7aa47605aefaa91568"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "74ace23a73ff3f4f37f9ed4a880fea8e"
server_key_iv: "c27d9f88a26641f5b664a8b36d94749e"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.677052 8289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:14:40.677909 8304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.678108 8289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:40.678207 8289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "5e551d3e46e24c7aa47605aefaa91568"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "74ace23a73ff3f4f37f9ed4a880fea8e"
server_key_iv: "c27d9f88a26641f5b664a8b36d94749e"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.678334 8289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:40.694984 8289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:40.696837 8289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:40.697019 8289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:40.697261 8289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:40.697609 8289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:40.697669 8289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.697703 8289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:40.697731 8289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.703328 8289 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:38873
I20260619 08:14:40.703379 8417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:38873 every 8 connection(s)
I20260619 08:14:40.703680 8289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:40.705801 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8289
I20260619 08:14:40.705914 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:40.706180 2527 external_mini_cluster.cc:1428] Setting key 5e86c81059d515651dd3c760a225c0a4
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:40.715178 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.704412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:34453 (local address 127.2.119.254:36703)
0619 08:14:40.704477 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.704479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.704720 (+ 241us) server_negotiation.cc:407] Connection header received
0619 08:14:40.705827 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.705829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.705839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.705952 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.706731 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.707303 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.708706 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.709033 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.709109 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.711753 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.711755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.711755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.711765 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.713455 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.713992 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.713993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.713994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.714059 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.714379 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.714380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.714381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.714537 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.714620 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.714986 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.715071 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:40.715325 8420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.704372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36703 (local address 127.2.119.194:34453)
0619 08:14:40.704557 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:40.704600 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:40.705668 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:40.706060 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:40.706065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:40.706430 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:40.706637 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.706642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.707402 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.707403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.708618 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.708620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.709140 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.709141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.709247 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.709922 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:40.709931 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:40.711631 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:40.713558 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.713565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.713567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.713901 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.714141 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.714141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.714142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.714266 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.714617 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:40.714622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:40.714749 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 08:14:40.714916 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 08:14:40.715081 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:14:40.715962 8418 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36703
I20260619 08:14:40.716049 8418 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:40.716187 8418 heartbeater.cc:511] Master 127.2.119.254:36703 requested a full tablet report, sending...
I20260619 08:14:40.716643 8092 ts_manager.cc:194] Registered new tserver with Master: 5e551d3e46e24c7aa47605aefaa91568 (127.2.119.194:38873)
I20260619 08:14:40.717100 8092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:34453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:40.736941 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:40.836100 8423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:40.836313 8423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:40.836336 8423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:40.837512 8423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:40.837553 8423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:40.837581 8423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:40.839018 8423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:40.839207 8423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:40.839381 8423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:40.844586 8431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.844591 8428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:40.844591 8429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:40.844928 8423 server_base.cc:1034] running on GCE node
I20260619 08:14:40.845155 8423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:40.845434 8423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:40.846590 8423 hybrid_clock.cc:648] HybridClock initialized: now 1781856880846577 us; error 26 us; skew 500 ppm
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.849059 8423 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:40.849555 8423 webserver.cc:460] Webserver started at http://127.2.119.195:36395/ using document root <none> and password file <none>
I20260619 08:14:40.849740 8423 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:40.849777 8423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:40.849867 8423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:40.850847 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "7eb9c6d5824d464f9f2a06e16adcd3b4"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "a5232632efd6c0a90acdef1f12414a90"
server_key_iv: "affc43fa4e682d7e2163df24e56233f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.851147 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "7eb9c6d5824d464f9f2a06e16adcd3b4"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "a5232632efd6c0a90acdef1f12414a90"
server_key_iv: "affc43fa4e682d7e2163df24e56233f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.852389 8423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:40.853050 8438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.853219 8423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:40.853368 8423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "7eb9c6d5824d464f9f2a06e16adcd3b4"
format_stamp: "Formatted at 2026-06-19 08:14:40 on dist-test-slave-d0b5"
server_key: "a5232632efd6c0a90acdef1f12414a90"
server_key_iv: "affc43fa4e682d7e2163df24e56233f4"
server_key_version: "encryptionkey@0"
I20260619 08:14:40.853452 8423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:40.874421 8423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:40.876626 8423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:40.876792 8423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:40.877054 8423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:40.877450 8423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:40.877509 8423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.877554 8423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:40.877581 8423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:40.883433 8423 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44559
I20260619 08:14:40.883490 8551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44559 every 8 connection(s)
I20260619 08:14:40.883844 8423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:40.892329 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8423
I20260619 08:14:40.892419 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:40.892663 2527 external_mini_cluster.cc:1428] Setting key 8f090c18c5fcea8320e7c535386b60ba
I20260619 08:14:40.893704 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.884633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:48775 (local address 127.2.119.254:36703)
0619 08:14:40.884716 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.884718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.884889 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:14:40.885914 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.885915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.885926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.885987 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.886557 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.887099 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.888078 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.888276 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.888339 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.890795 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.890797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.890797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.890804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.892080 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.892525 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.892527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.892528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.892593 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.892922 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.892923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.892923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.893074 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.893164 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.893510 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.893612 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:14:40.893939 8554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.884587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36703 (local address 127.2.119.195:48775)
0619 08:14:40.884762 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:40.884803 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:40.885818 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:40.886040 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:40.886043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:40.886290 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:40.886476 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.886479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.887191 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.887191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.887996 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:40.887998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.888357 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.888358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:40.888458 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.889065 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:40.889076 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:40.890671 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:40.892156 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.892162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.892164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.892428 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.892666 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:40.892667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:40.892667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:40.892817 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:40.893145 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:40.893151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:40.893369 (+ 218us) client_negotiation.cc:769] Sending connection context
0619 08:14:40.893487 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 08:14:40.893656 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260619 08:14:40.894551 8552 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36703
I20260619 08:14:40.894658 8552 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:40.894850 8552 heartbeater.cc:511] Master 127.2.119.254:36703 requested a full tablet report, sending...
I20260619 08:14:40.895248 8092 ts_manager.cc:194] Registered new tserver with Master: 7eb9c6d5824d464f9f2a06e16adcd3b4 (127.2.119.195:44559)
I20260619 08:14:40.895653 8092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:48775
I20260619 08:14:40.904541 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/test-admin.keytab.
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.922160 2527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 08:14:40.923772 8559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:40.922998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.2.119.193:36959)
0619 08:14:40.923197 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.923202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.923213 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:40.923239 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.923240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.923254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.923318 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.923665 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34644: BlockingRecv error: recv got EOF from 127.0.0.1:34644 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":114,"threads_started":1}
W20260619 08:14:40.923866 2527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.924850 2527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.925952 2527 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:40.934254 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.926533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.2.119.193:36959)
0619 08:14:40.926595 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.926596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.926624 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:14:40.926715 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.926716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.926729 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.926773 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.927171 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.927910 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.928687 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.929017 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.929098 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.931404 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.931413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.931413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.931421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.933018 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.933410 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.933414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.933414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.933473 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.933686 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.933687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.933687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.933826 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.933999 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.934056 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.934136 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 08:14:40.936136 8559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:40.935562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.2.119.193:36959)
0619 08:14:40.935604 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.935606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.935691 (+ 85us) server_negotiation.cc:407] Connection header received
0619 08:14:40.935748 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.935748 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.935760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.935802 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.936084 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34656: BlockingRecv error: recv got EOF from 127.0.0.1:34656 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 08:14:40.936321 2527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.937486 2527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.938414 2527 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:40.945593 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.939049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.2.119.193:36959)
0619 08:14:40.939086 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.939090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.939103 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:14:40.939121 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.939121 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.939130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.939183 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.939533 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.940012 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.940720 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.940917 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.940971 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.943207 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.943208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.943208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.943215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.944430 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.944817 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.944817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.944818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.944870 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.945064 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.945064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.945064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.945158 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.945230 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.945460 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.945527 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:14:40.946960 8559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:40.946605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.2.119.193:36959)
0619 08:14:40.946640 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.946641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.946668 (+ 27us) server_negotiation.cc:407] Connection header received
0619 08:14:40.946698 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.946699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.946706 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.946744 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.946928 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34664: BlockingRecv error: recv got EOF from 127.0.0.1:34664 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 08:14:40.947227 2527 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781856856097588-2527-0/krb5kdc/krb5cc' of type FILE
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.948182 2527 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:40.949011 2527 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 08:14:40 dist-test-slave-d0b5 krb5kdc[8048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:14:40.956220 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:40.949566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.2.119.193:36959)
0619 08:14:40.949605 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:40.949607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:40.949635 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:14:40.949706 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:40.949706 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:40.949715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:40.949783 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:40.950228 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.950698 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.951395 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:40.951587 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:40.951666 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:40.953665 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:40.953666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:40.953667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:40.953673 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:40.954963 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.955385 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.955387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.955387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.955450 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:40.955706 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:40.955707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:40.955707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:40.955857 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:40.955944 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:40.956048 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 08:14:40.956120 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:40.956815 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8155
I20260619 08:14:40.961475 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8289
I20260619 08:14:40.966514 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8423
I20260619 08:14:40.971012 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8061
2026-06-19T08:14:40Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1921 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:41 dist-test-slave-d0b5 krb5kdc[8572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:41 dist-test-slave-d0b5 krb5kdc[8572](info): set up 2 sockets
Jun 19 08:14:41 dist-test-slave-d0b5 krb5kdc[8572](info): commencing operation
krb5kdc: starting...
W20260619 08:14:42.059263 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab.
Entry for principal oryx/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab.
I20260619 08:14:42.188669 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41109
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab
--principal=oryx/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:42.270481 8585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:42.270701 8585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:42.270725 8585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:42.271858 8585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:42.271891 8585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:42.271905 8585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:42.271919 8585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:42.271930 8585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:42.273486 8585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab
--principal=oryx/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:42.273694 8585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:42.273875 8585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:42.278878 8593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.278878 8591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.279074 8590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:42.279212 8585 server_base.cc:1034] running on GCE node
I20260619 08:14:42.279470 8585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:42.279738 8585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:42.280902 8585 hybrid_clock.cc:648] HybridClock initialized: now 1781856882280874 us; error 47 us; skew 500 ppm
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:42.286652 8585 init.cc:377] Logged in from keytab as oryx/127.2.119.254@KRBTEST.COM (short username oryx)
I20260619 08:14:42.287050 8585 webserver.cc:460] Webserver started at http://127.2.119.254:36881/ using document root <none> and password file <none>
I20260619 08:14:42.287240 8585 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:42.287279 8585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:42.287375 8585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:42.288379 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "647de3d06aec439e8eda74ec862139c4"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "933ce79e0d574f08da3016fd4f645cb2"
server_key_iv: "efdb36f091acf4df881c2809cfde087c"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.288688 8585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "647de3d06aec439e8eda74ec862139c4"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "933ce79e0d574f08da3016fd4f645cb2"
server_key_iv: "efdb36f091acf4df881c2809cfde087c"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.290081 8585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:42.290859 8600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.291069 8585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:42.291152 8585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "647de3d06aec439e8eda74ec862139c4"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "933ce79e0d574f08da3016fd4f645cb2"
server_key_iv: "efdb36f091acf4df881c2809cfde087c"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.291208 8585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:42.333700 8585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:42.336062 8585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:42.336313 8585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:42.340353 8585 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41109
I20260619 08:14:42.340443 8652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41109 every 8 connection(s)
I20260619 08:14:42.340787 8585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:42.341885 8653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:42.344165 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8585
I20260619 08:14:42.344291 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:42.344514 2527 external_mini_cluster.cc:1428] Setting key b916cdb4277d6522f01a3cd7654e7698
I20260619 08:14:42.344616 8653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Bootstrap starting.
I20260619 08:14:42.345657 8653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:42.345965 8653 log.cc:824] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:42.346693 8653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: No bootstrap required, opened a new log
I20260619 08:14:42.348413 8653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } }
I20260619 08:14:42.348573 8653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:42.348619 8653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647de3d06aec439e8eda74ec862139c4, State: Initialized, Role: FOLLOWER
I20260619 08:14:42.348745 8653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } }
I20260619 08:14:42.348857 8653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:42.348907 8653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:42.348954 8653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:42.349561 8653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } }
I20260619 08:14:42.349684 8653 leader_election.cc:302] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 647de3d06aec439e8eda74ec862139c4; no voters:
I20260619 08:14:42.349885 8653 leader_election.cc:288] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:42.349913 8658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:42.350082 8658 raft_consensus.cc:686] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 LEADER]: Becoming Leader. State: Replica: 647de3d06aec439e8eda74ec862139c4, State: Running, Role: LEADER
I20260619 08:14:42.350193 8658 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } }
I20260619 08:14:42.350227 8653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:42.350549 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } }
I20260619 08:14:42.350627 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:42.350767 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647de3d06aec439e8eda74ec862139c4. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } }
I20260619 08:14:42.350838 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:42.350972 8664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:42.351471 8664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:42.353459 8664 catalog_manager.cc:1347] Generated new cluster ID: df75f46532fb4a7da806e75696f4c5cc
I20260619 08:14:42.353523 8664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:42.355757 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.345002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.2.119.254:41109)
0619 08:14:42.345193 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:42.345204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:42.345245 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:14:42.345834 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:42.345841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:42.345889 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:42.346039 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:42.346667 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.347595 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.348362 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.348852 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.348976 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.352000 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:42.352004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:42.352004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:42.352026 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:42.354104 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.354485 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.354488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.354488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.354599 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.354814 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.354815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.354815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.355120 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:42.355269 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:42.355376 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 08:14:42.355528 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260619 08:14:42.371338 8664 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:42.372167 8664 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:42.377821 8664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Generated new TSK 0
I20260619 08:14:42.378006 8664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
I20260619 08:14:42.410625 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:42.493234 8679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:42.493517 8679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:42.493542 8679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:42.494787 8679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:42.494890 8679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:42.494933 8679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:42.496521 8679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:42.496798 8679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:42.497038 8679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:42.502380 8687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.502369 8684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.502369 8685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:42.502688 8679 server_base.cc:1034] running on GCE node
I20260619 08:14:42.502928 8679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:42.503173 8679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:42.504348 8679 hybrid_clock.cc:648] HybridClock initialized: now 1781856882504351 us; error 49 us; skew 500 ppm
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:42.507110 8679 init.cc:377] Logged in from keytab as oryx/127.2.119.193@KRBTEST.COM (short username oryx)
I20260619 08:14:42.507586 8679 webserver.cc:460] Webserver started at http://127.2.119.193:45467/ using document root <none> and password file <none>
I20260619 08:14:42.507910 8679 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:42.507997 8679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:42.508162 8679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:42.509279 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "4b9b52c47e2e4d5082a28ae05789bd6e"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "761285e6d29da8d515218d9846b9f969"
server_key_iv: "54d41e2cdb94b12dca52231f6db39053"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.509756 8679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "4b9b52c47e2e4d5082a28ae05789bd6e"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "761285e6d29da8d515218d9846b9f969"
server_key_iv: "54d41e2cdb94b12dca52231f6db39053"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.511245 8679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:14:42.512032 8694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.512253 8679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:42.512363 8679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "4b9b52c47e2e4d5082a28ae05789bd6e"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "761285e6d29da8d515218d9846b9f969"
server_key_iv: "54d41e2cdb94b12dca52231f6db39053"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.512457 8679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:42.540158 8679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:42.542562 8679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:42.542744 8679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:42.543006 8679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:42.543346 8679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:42.543406 8679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.543442 8679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:42.543470 8679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.549399 8679 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:45861
I20260619 08:14:42.549458 8807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:45861 every 8 connection(s)
I20260619 08:14:42.549775 8679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:42.555999 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8679
I20260619 08:14:42.556087 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:42.556315 2527 external_mini_cluster.cc:1428] Setting key 5c38afccf8b782ff3f0ba7b26c93d343
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:42.560372 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.550561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:53287 (local address 127.2.119.254:41109)
0619 08:14:42.550610 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:42.550612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:42.550836 (+ 224us) server_negotiation.cc:407] Connection header received
0619 08:14:42.551463 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:42.551464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:42.551473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:42.551534 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:42.552846 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.553452 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.554436 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.554644 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.554707 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.557005 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:42.557007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:42.557007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:42.557015 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:42.558776 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.559219 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.559220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.559220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.559284 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.559576 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.559577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.559577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.559728 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:42.559801 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:42.560139 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 08:14:42.560230 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:42.560384 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.550536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:53287)
0619 08:14:42.550702 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:42.550744 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:42.551381 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:42.551608 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:42.551612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:42.552394 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:42.552594 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.552599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.553542 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.553543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.554336 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.554338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.554727 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.554728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.554849 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.555384 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:42.555393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:42.556894 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:42.558911 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.558917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.558918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.559137 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.559387 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.559388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.559388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.559496 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.559816 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:42.559822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:42.559996 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 08:14:42.560088 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:14:42.560207 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260619 08:14:42.561008 8808 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:42.561129 8808 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:42.561395 8808 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:42.562050 8617 ts_manager.cc:194] Registered new tserver with Master: 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
I20260619 08:14:42.563097 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:53287
WARNING: no policy specified for oryx/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
I20260619 08:14:42.584859 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:42.666265 8813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:42.666563 8813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:42.666601 8813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:42.667740 8813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:42.667814 8813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:42.667857 8813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:42.669288 8813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:42.669588 8813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:42.669842 8813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:42.674839 8821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.674815 8818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.674786 8819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:42.674997 8813 server_base.cc:1034] running on GCE node
I20260619 08:14:42.675179 8813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:42.675434 8813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:42.676571 8813 hybrid_clock.cc:648] HybridClock initialized: now 1781856882676555 us; error 43 us; skew 500 ppm
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:42.679157 8813 init.cc:377] Logged in from keytab as oryx/127.2.119.194@KRBTEST.COM (short username oryx)
I20260619 08:14:42.679608 8813 webserver.cc:460] Webserver started at http://127.2.119.194:44555/ using document root <none> and password file <none>
I20260619 08:14:42.679788 8813 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:42.679827 8813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:42.679917 8813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:42.680924 8813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "d34658d33b794e7ca29feabb708aba5c"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "3d1f6bbd34e41c2479d401c196076c74"
server_key_iv: "db78678771e8609ddc49ad0252cc82ee"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.681226 8813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "d34658d33b794e7ca29feabb708aba5c"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "3d1f6bbd34e41c2479d401c196076c74"
server_key_iv: "db78678771e8609ddc49ad0252cc82ee"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.682631 8813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:42.683372 8828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.683629 8813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:42.683713 8813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "d34658d33b794e7ca29feabb708aba5c"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "3d1f6bbd34e41c2479d401c196076c74"
server_key_iv: "db78678771e8609ddc49ad0252cc82ee"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.683770 8813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:42.697149 8813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:42.699282 8813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:42.699391 8813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:42.699626 8813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:42.699970 8813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:42.700014 8813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.700040 8813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:42.700052 8813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.705999 8813 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:39185
I20260619 08:14:42.706115 8941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:39185 every 8 connection(s)
I20260619 08:14:42.706411 8813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:42.709977 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8813
I20260619 08:14:42.710124 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:42.710305 2527 external_mini_cluster.cc:1428] Setting key 173541971ece360e53fe2bebbc2d465e
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:42.717789 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.707317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:50503 (local address 127.2.119.254:41109)
0619 08:14:42.707403 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:42.707406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:42.707609 (+ 203us) server_negotiation.cc:407] Connection header received
0619 08:14:42.708230 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:42.708232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:42.708247 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:42.708299 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:42.709399 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.709974 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.711402 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.711627 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.711725 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.714364 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:42.714366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:42.714366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:42.714375 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:42.716046 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.716668 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.716669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.716670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.716737 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.717068 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.717069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.717069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.717188 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:42.717242 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:42.717659 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 08:14:42.717725 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:14:42.718182 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.707281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:50503)
0619 08:14:42.707451 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:42.707505 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:42.708100 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:42.708388 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:42.708391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:42.709017 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:42.709226 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.709231 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.710085 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.710086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.711279 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.711282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.711736 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.711737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.711877 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.712515 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:42.712528 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:42.714257 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:42.716138 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.716146 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.716149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.716541 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.716816 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.716817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.716818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.716974 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.717268 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:42.717275 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:42.717494 (+ 219us) client_negotiation.cc:769] Sending connection context
0619 08:14:42.717622 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 08:14:42.717815 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260619 08:14:42.718777 8942 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:42.718895 8942 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:42.719122 8942 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:42.719615 8617 ts_manager.cc:194] Registered new tserver with Master: d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
WARNING: no policy specified for oryx/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:14:42.720043 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:50503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
I20260619 08:14:42.741628 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:42.827556 8947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:42.827764 8947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:42.827787 8947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:42.829008 8947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:42.829051 8947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:42.829079 8947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:42.830616 8947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:42.830861 8947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:42.831122 8947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:42.836416 8953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.836462 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:42.836421 8952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:42.836688 8947 server_base.cc:1034] running on GCE node
I20260619 08:14:42.836884 8947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:42.837121 8947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:42.838285 8947 hybrid_clock.cc:648] HybridClock initialized: now 1781856882838259 us; error 63 us; skew 500 ppm
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:42.840842 8947 init.cc:377] Logged in from keytab as oryx/127.2.119.195@KRBTEST.COM (short username oryx)
I20260619 08:14:42.841346 8947 webserver.cc:460] Webserver started at http://127.2.119.195:35059/ using document root <none> and password file <none>
I20260619 08:14:42.841579 8947 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:42.841645 8947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:42.841777 8947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:42.842828 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "4f2db21736e11d71a32def8238295e4a"
server_key_iv: "a74e5c5b59746bf204b6bbe4737b73f2"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.843191 8947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "4f2db21736e11d71a32def8238295e4a"
server_key_iv: "a74e5c5b59746bf204b6bbe4737b73f2"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.844568 8947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:42.845599 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.845778 8947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:42.845891 8947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "4f2db21736e11d71a32def8238295e4a"
server_key_iv: "a74e5c5b59746bf204b6bbe4737b73f2"
server_key_version: "encryptionkey@0"
I20260619 08:14:42.845979 8947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:42.865693 8947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:42.868268 8947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:42.868462 8947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:42.868711 8947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:42.869095 8947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:42.869154 8947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.869200 8947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:42.869225 8947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:42.874867 8947 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33301
I20260619 08:14:42.875007 9075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33301 every 8 connection(s)
I20260619 08:14:42.875293 8947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:42.877002 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 8947
I20260619 08:14:42.877116 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:42.877336 2527 external_mini_cluster.cc:1428] Setting key 6507983d1ccb375b8907c5a812037460
Jun 19 08:14:42 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:42.885203 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.876173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:57295 (local address 127.2.119.254:41109)
0619 08:14:42.876246 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:42.876248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:42.876417 (+ 169us) server_negotiation.cc:407] Connection header received
0619 08:14:42.877129 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:42.877131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:42.877144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:42.877213 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:42.878520 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.879041 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.880117 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.880308 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.880367 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.882576 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:42.882577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:42.882577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:42.882584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:42.883959 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.884359 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.884360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.884361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.884419 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:42.884640 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:42.884640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:42.884641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:42.884745 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:42.884798 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:42.885068 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 08:14:42.885138 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:14:42.885285 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:42.876064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:57295)
0619 08:14:42.876282 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:42.876336 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:42.876893 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:42.877388 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:42.877396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:42.878187 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:42.878382 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.878387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.879139 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.879140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.880035 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:42.880038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:42.880382 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:42.880382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:42.880481 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:42.881007 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:42.881017 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:42.882491 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:42.884048 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.884054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.884055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.884267 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.884489 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:42.884489 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:42.884489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:42.884583 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:42.884801 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:42.884804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:42.884925 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 08:14:42.885010 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:14:42.885124 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":94,"threads_started":1}
I20260619 08:14:42.885931 9076 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:42.886018 9076 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:42.886152 9076 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:42.886515 8617 ts_manager.cc:194] Registered new tserver with Master: 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301)
I20260619 08:14:42.886953 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:57295
I20260619 08:14:42.889266 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.119.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.253.keytab.
Entry for principal oryx/127.2.119.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.253.keytab.
I20260619 08:14:42.916745 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:39527
--webserver_interface=127.2.119.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.253.keytab
--principal=oryx/127.2.119.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41109,127.2.119.253:39527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:43.002477 9081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:43.002688 9081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:43.002712 9081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:43.003895 9081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:43.003937 9081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:43.003952 9081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:43.003965 9081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:43.003978 9081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:43.005549 9081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41109,127.2.119.253:39527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.253.keytab
--principal=oryx/127.2.119.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:39527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.119.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
W20260619 08:14:43.005656 9081 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.119.254:41109,127.2.119.253:39527'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 08:14:43.005841 9081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:43.006079 9081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.011166 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.011322 9089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.011269 9081 server_base.cc:1034] running on GCE node
W20260619 08:14:43.011265 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.011634 9081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.011991 9081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.013195 9081 hybrid_clock.cc:648] HybridClock initialized: now 1781856883013163 us; error 46 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.015765 9081 init.cc:377] Logged in from keytab as oryx/127.2.119.253@KRBTEST.COM (short username oryx)
I20260619 08:14:43.016175 9081 webserver.cc:460] Webserver started at http://127.2.119.253:45677/ using document root <none> and password file <none>
I20260619 08:14:43.016366 9081 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.016450 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.016563 9081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:43.017755 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/instance:
uuid: "d1ee07da405c45eb8499bd96215fe128"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "a613e7e01116702a70faf4db40fa6454"
server_key_iv: "633cf9902ec078bc5f140b50b411e602"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.018054 9081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal/instance:
uuid: "d1ee07da405c45eb8499bd96215fe128"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "a613e7e01116702a70faf4db40fa6454"
server_key_iv: "633cf9902ec078bc5f140b50b411e602"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.019341 9081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:14:43.020112 9096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.020345 9081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:43.020450 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "d1ee07da405c45eb8499bd96215fe128"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "a613e7e01116702a70faf4db40fa6454"
server_key_iv: "633cf9902ec078bc5f140b50b411e602"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.020537 9081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.045814 9081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.047683 9081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:43.047850 9081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.052227 9081 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.253:39527
I20260619 08:14:43.052261 9148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.253:39527 every 8 connection(s)
I20260619 08:14:43.052620 9081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
I20260619 08:14:43.053694 9149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:43.055236 9149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.253@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:43.061971 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9081
I20260619 08:14:43.064481 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.055940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.2.119.254:41109)
0619 08:14:43.056002 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.056003 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.056249 (+ 246us) server_negotiation.cc:407] Connection header received
0619 08:14:43.056665 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.056666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.056676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.056724 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.057270 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.057807 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.058746 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.058934 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.058988 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.061056 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.061058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.061058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.061064 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.062794 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.063335 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.063336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.063336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.063387 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.063745 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.063745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.063746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.063935 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.064007 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.064303 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.064384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:43.064939 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.055951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48288)
0619 08:14:43.056117 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.056151 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.056578 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.056782 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.056785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.057016 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.057198 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.057201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.057882 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.057883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.058663 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.058664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.059005 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.059006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.059104 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.059596 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.059605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.060950 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.062899 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.062906 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.062908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.063253 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.063461 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.063462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.063462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.063643 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.064024 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.064030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.064162 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.064259 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.064729 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":266,"thread_start_us":86,"threads_started":1}
I20260619 08:14:43.065712 9149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.253@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
I20260619 08:14:43.072285 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.062541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.2.119.253:39527)
0619 08:14:43.062713 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.062718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.062768 (+ 50us) server_negotiation.cc:407] Connection header received
0619 08:14:43.062791 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.062793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.062807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.062877 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.063745 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.064699 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.065658 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.066170 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.066243 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.069215 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.069222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.069223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.069230 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.071067 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.071481 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.071484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.071485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.071548 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.071775 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.071776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.071776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.071900 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.072016 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.072137 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.072217 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260619 08:14:43.073930 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.065949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57650)
0619 08:14:43.065994 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.066004 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.066075 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.066312 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.066313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.066450 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.066519 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.066520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.067329 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.067331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.068221 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.068223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.068751 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.068752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.068866 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.069259 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.069264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.070846 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.073153 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.073153 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.073154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.073382 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.073517 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.073518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.073518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.073560 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.073705 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.073706 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.073735 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.073778 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.073842 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:14:43.073999 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.066030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.2.119.253:39527)
0619 08:14:43.066169 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.066170 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.066179 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:43.066198 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.066199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.066206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.066283 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.066579 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.067248 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.068297 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.068629 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.068701 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.070931 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.070933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.070933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.070939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.073092 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.073435 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.073436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.073437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.073479 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.073592 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.073593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.073593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.073671 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.073714 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.073791 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.073881 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260619 08:14:43.075203 9149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Bootstrap starting.
I20260619 08:14:43.075807 9149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:43.076045 9149 log.cc:824] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:43.076694 9149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: No bootstrap required, opened a new log
I20260619 08:14:43.078135 9149 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } }
I20260619 08:14:43.078254 9149 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:43.078277 9149 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ee07da405c45eb8499bd96215fe128, State: Initialized, Role: FOLLOWER
I20260619 08:14:43.078400 9149 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } }
I20260619 08:14:43.078646 9156 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } } }
I20260619 08:14:43.078715 9149 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:43.078727 9156 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:14:43.080577 9167 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:14:43.080636 9167 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:14:43.082006 9081 master_runner.cc:428] Detected that this master d1ee07da405c45eb8499bd96215fe128 is joining an existing cluster
I20260619 08:14:43.082080 9081 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.119.253:39527
I20260619 08:14:43.090590 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.082946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.2.119.254:41109)
0619 08:14:43.082986 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.082988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.083183 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:14:43.083271 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.083272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.083283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.083339 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.083718 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.084533 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.086243 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.086525 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.086585 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.087661 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.087662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.087662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.087669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.089272 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089731 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.089733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.089733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.089789 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.090027 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.090028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.090028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.090158 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.090191 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.090409 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.090488 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:43.090689 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.083182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.2.119.253:39527)
0619 08:14:43.083217 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.083218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.083280 (+ 62us) server_negotiation.cc:407] Connection header received
0619 08:14:43.083366 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.083367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.083375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.083401 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.083819 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.084526 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.085634 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.085993 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.086060 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.087291 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.087293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.087293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.087300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.089420 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089821 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.089822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.089822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.089874 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.090095 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.090096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.090096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.090229 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.090298 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.090438 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.090571 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":20}
I20260619 08:14:43.090911 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.082881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48300)
0619 08:14:43.083114 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.083125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.083212 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.083345 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.083346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.083552 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.083631 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.083633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.085391 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.085392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.086164 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.086165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.086636 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.086637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.086699 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.087140 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.087145 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.087552 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.089379 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.089380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.089653 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.089861 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.089862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.089940 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.090228 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.090229 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.090293 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.090718 (+ 425us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.090827 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
I20260619 08:14:43.091095 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.083107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57658)
0619 08:14:43.083224 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.083234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.083302 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.083456 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.083456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.083681 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.083741 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.083743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.084613 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.084614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.085552 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.085555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.086069 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.086070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.086146 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.086703 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.086708 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.087200 (+ 492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.089518 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.089519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.089750 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.089953 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.089954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.089954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.090012 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.090303 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.090303 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.090340 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.090414 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.090507 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.254@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
I20260619 08:14:43.100453 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.093139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57674)
0619 08:14:43.093414 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.093445 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.093507 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.093663 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.093667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.093976 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.094029 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.094031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.094930 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.094931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.095540 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.095542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.095995 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.095995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.096074 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.096514 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.096519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.097859 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.099495 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.099501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.099503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.099685 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.099879 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.099879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.099879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.099930 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.100172 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.100177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.100219 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.100287 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.100370 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":63,"threads_started":1}
I20260619 08:14:43.100502 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.093214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.2.119.253:39527)
0619 08:14:43.093285 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.093287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.093511 (+ 224us) server_negotiation.cc:407] Connection header received
0619 08:14:43.093580 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.093581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.093610 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.093658 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.094101 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.094839 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.095620 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.095924 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.095991 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.097939 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.097940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.097941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.097948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.099416 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.099757 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.099758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.099758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.099814 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.099988 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.099989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.099989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.100110 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.100173 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.100330 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.100401 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:14:43.101066 8617 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.119.253:39527: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d1ee07da405c45eb8499bd96215fe128"
member_type: NON_VOTER
last_known_addr {
host: "127.2.119.253"
port: 39527
}
attrs {
promote: true
}
}
dest_uuid: "647de3d06aec439e8eda74ec862139c4"
cas_config_opid_index: -1
I20260619 08:14:43.101413 8617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } }
I20260619 08:14:43.101910 9176 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d1ee07da405c45eb8499bd96215fe128 (127.2.119.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } }
I20260619 08:14:43.102238 9123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:43.102885 9176 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.119.253:39527
I20260619 08:14:43.102978 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } }
I20260619 08:14:43.103056 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.103134 9081 master.cc:537] Master@127.2.119.253:39527 shutting down...
I20260619 08:14:43.103219 9123 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 08:14:43.103618 9177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:14:43.104297 8603 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 08:14:43.104775 8603 consensus_peers.cc:487] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 -> Peer d1ee07da405c45eb8499bd96215fe128 (127.2.119.253:39527): Couldn't send request to peer d1ee07da405c45eb8499bd96215fe128. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:14:43.105856 9081 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:43.105948 9081 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:43.105999 9081 tablet_replica.cc:330] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: stopping tablet replica
I20260619 08:14:43.118551 9081 master.cc:552] Master@127.2.119.253:39527 shutdown complete.
I20260619 08:14:43.118602 9081 master_runner.cc:305] Clearing existing system tablet
I20260619 08:14:43.118745 9081 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.118796 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.119423 9081 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:43.119917 9181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.120043 9081 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.120100 9081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.120121 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "d1ee07da405c45eb8499bd96215fe128"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "a613e7e01116702a70faf4db40fa6454"
server_key_iv: "633cf9902ec078bc5f140b50b411e602"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.120736 9081 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:43.122246 9081 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:14:43.122304 9081 log.cc:1196] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 08:14:43.122499 9081 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Deleting consensus metadata
I20260619 08:14:43.122633 9081 master_runner.cc:315] Copying system tablet from 127.2.119.254:41109
I20260619 08:14:43.123256 9081 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: tablet copy: Beginning tablet copy session from remote peer at address 127.2.119.254:41109
I20260619 08:14:43.129599 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.123606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.2.119.254:41109)
0619 08:14:43.123647 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.123649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.123724 (+ 75us) server_negotiation.cc:407] Connection header received
0619 08:14:43.123792 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.123793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.123801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.123846 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.124206 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.124704 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.125812 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.126042 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.126100 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.127102 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.127103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.127103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.127109 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.128503 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.128891 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.128892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.128892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.128945 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.129136 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.129137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.129137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.129241 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.129330 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.129471 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.129536 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:43.129666 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.123529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48312)
0619 08:14:43.123663 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.123673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.123732 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.123861 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.123861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.124062 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.124136 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.124138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.124781 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.124782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.125725 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.125728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.126117 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.126118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.126183 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.126620 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.126625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.127017 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.128589 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.128590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.128591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.128814 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.129009 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.129010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.129010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.129074 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.129323 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.129324 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.129372 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.129450 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.129552 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260619 08:14:43.129971 8637 tablet_copy_service.cc:140] P 647de3d06aec439e8eda74ec862139c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d1ee07da405c45eb8499bd96215fe128 ({username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:48312)
I20260619 08:14:43.130076 8637 tablet_copy_service.cc:161] P 647de3d06aec439e8eda74ec862139c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d1ee07da405c45eb8499bd96215fe128 at {username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:48312: session id = d1ee07da405c45eb8499bd96215fe128-00000000000000000000000000000000
I20260619 08:14:43.130918 8637 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:14:43.131589 9081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:14:43.132550 9081 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: tablet copy: Starting download of 0 data blocks...
I20260619 08:14:43.132710 9081 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: tablet copy: Starting download of 1 WAL segments...
I20260619 08:14:43.133704 9081 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:14:43.134465 8637 tablet_copy_service.cc:342] P 647de3d06aec439e8eda74ec862139c4: Request end of tablet copy session d1ee07da405c45eb8499bd96215fe128-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:48312
I20260619 08:14:43.134536 8637 tablet_copy_service.cc:433] P 647de3d06aec439e8eda74ec862139c4: ending tablet copy session d1ee07da405c45eb8499bd96215fe128-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d1ee07da405c45eb8499bd96215fe128
I20260619 08:14:43.136715 9081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.137900 9188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.137996 9191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.138134 9189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.138612 9081 server_base.cc:1034] running on GCE node
I20260619 08:14:43.138718 9081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.138852 9081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.139977 9081 hybrid_clock.cc:648] HybridClock initialized: now 1781856883139968 us; error 23 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.141374 9081 init.cc:377] Logged in from keytab as oryx/127.2.119.253@KRBTEST.COM (short username oryx)
I20260619 08:14:43.141661 9081 webserver.cc:460] Webserver started at http://127.2.119.253:45893/ using document root <none> and password file <none>
I20260619 08:14:43.141794 9081 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.141834 9081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.142421 9081 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.142939 9198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.143071 9081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.143118 9081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "d1ee07da405c45eb8499bd96215fe128"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "a613e7e01116702a70faf4db40fa6454"
server_key_iv: "633cf9902ec078bc5f140b50b411e602"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.143172 9081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.166131 9081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.167994 9081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.170603 9081 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.253:39527
I20260619 08:14:43.170629 9250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.253:39527 every 8 connection(s)
I20260619 08:14:43.170940 9081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
I20260619 08:14:43.171837 9251 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:14:43.172488 9251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Bootstrap starting.
I20260619 08:14:43.175359 9251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:14:43.175611 9251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Bootstrap complete.
I20260619 08:14:43.175807 9251 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } }
I20260619 08:14:43.175894 9251 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d1ee07da405c45eb8499bd96215fe128, State: Initialized, Role: LEARNER
I20260619 08:14:43.175937 9251 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } }
I20260619 08:14:43.176126 9251 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:43.176090 9255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } }
I20260619 08:14:43.177313 9255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:14:43.177906 9266 catalog_manager.cc:1259] Loaded cluster ID: df75f46532fb4a7da806e75696f4c5cc
I20260619 08:14:43.177949 9266 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: loading cluster ID for follower catalog manager: success
I20260619 08:14:43.178774 9266 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: acquiring CA information for follower catalog manager: success
I20260619 08:14:43.179061 9266 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:14:43.181907 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.174024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.2.119.253:39527)
0619 08:14:43.174147 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.174149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.174158 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:43.174183 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.174184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.174195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.174282 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.174971 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.176035 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.177080 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.177412 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.178268 (+ 856us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.179331 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.179332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.179333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.179339 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.180733 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.181175 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.181176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.181176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.181229 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.181460 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.181461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.181461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.181564 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.181637 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.181775 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.181841 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.2.119.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.252.keytab.
Entry for principal oryx/127.2.119.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.252.keytab.
I20260619 08:14:43.209108 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:36871
--webserver_interface=127.2.119.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.252.keytab
--principal=oryx/127.2.119.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:43.295529 9269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:43.295737 9269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:43.295759 9269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:43.296860 9269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:43.296895 9269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:43.296909 9269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:43.296923 9269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:43.296936 9269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:43.298476 9269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.252.keytab
--principal=oryx/127.2.119.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:36871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.119.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:43.298663 9269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:43.298825 9269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.303979 9275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.304066 9269 server_base.cc:1034] running on GCE node
W20260619 08:14:43.304047 9277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.304042 9274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.304435 9269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.304714 9269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.305866 9269 hybrid_clock.cc:648] HybridClock initialized: now 1781856883305846 us; error 33 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.308482 9269 init.cc:377] Logged in from keytab as oryx/127.2.119.252@KRBTEST.COM (short username oryx)
I20260619 08:14:43.309038 9269 webserver.cc:460] Webserver started at http://127.2.119.252:42627/ using document root <none> and password file <none>
I20260619 08:14:43.309283 9269 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.309401 9269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.309583 9269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:43.310746 9269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/instance:
uuid: "c8a4dc2bcea244a2909faf5701933ae2"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "cbc15ce5818fa24ef98e55ed8790c6a1"
server_key_iv: "fad0881e0c4a1c90923e6b94dccf04ab"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.311151 9269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal/instance:
uuid: "c8a4dc2bcea244a2909faf5701933ae2"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "cbc15ce5818fa24ef98e55ed8790c6a1"
server_key_iv: "fad0881e0c4a1c90923e6b94dccf04ab"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.312621 9269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:43.313560 9284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.313766 9269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:43.313894 9269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "c8a4dc2bcea244a2909faf5701933ae2"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "cbc15ce5818fa24ef98e55ed8790c6a1"
server_key_iv: "fad0881e0c4a1c90923e6b94dccf04ab"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.313973 9269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.335554 9269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.337845 9269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:43.338057 9269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.342366 9269 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.252:36871
I20260619 08:14:43.342391 9336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.252:36871 every 8 connection(s)
I20260619 08:14:43.342806 9269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
I20260619 08:14:43.343861 9337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:43.344434 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9269
I20260619 08:14:43.345947 9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.252@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
I20260619 08:14:43.354678 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.346731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.2.119.254:41109)
0619 08:14:43.346778 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.346780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.346901 (+ 121us) server_negotiation.cc:407] Connection header received
0619 08:14:43.346977 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.346978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.346989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.347034 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.347514 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.348002 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.348884 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.349103 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.349175 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.351432 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.351433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.351434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.351441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.353054 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.353608 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.353610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.353610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.353669 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.353885 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.353886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.353886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.354045 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.354120 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.354486 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.354566 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 08:14:43.354863 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.346625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48324)
0619 08:14:43.346788 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.346829 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.346925 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.347071 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.347074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.347329 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.347419 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.347424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.348109 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.348110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.348796 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.348798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.349197 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.349198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.349249 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.349615 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.349626 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.351329 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.353132 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.353139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.353141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.353537 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.353739 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.353740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.353740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.353826 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.354115 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.354119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.354286 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.354438 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.354581 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260619 08:14:43.355438 9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:14:43.355558 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.345108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56574 (local address 127.2.119.252:36871)
0619 08:14:43.345277 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.345284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.345359 (+ 75us) server_negotiation.cc:407] Connection header received
0619 08:14:43.345769 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.345775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.345820 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.345936 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.346726 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.347618 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.348329 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.348791 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.348908 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.352171 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.352179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.352179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.352186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.354125 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.354489 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.354492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.354493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.354556 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.354789 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.354790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.354790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.355144 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.355273 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.355389 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.355448 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.252@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
I20260619 08:14:43.361833 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.355698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57688)
0619 08:14:43.355732 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.355743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.355779 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.356004 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.356004 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.356106 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.356135 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.356136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.356829 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.356830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.357413 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.357415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.357778 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.357779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.357826 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.358268 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.358272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.359408 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.360840 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.360841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.360842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.361047 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.361222 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.361223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.361223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.361272 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.361605 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.361606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.361641 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.361705 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.361775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 08:14:43.361881 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.355791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.2.119.253:39527)
0619 08:14:43.355834 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.355836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.355844 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:43.355869 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.355869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.355879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.356047 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.356207 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.356733 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.357492 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.357687 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.357748 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.359510 (+ 1762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.359511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.359511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.359518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.360758 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.361115 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.361115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.361116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.361164 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.361408 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.361409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.361409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.361515 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.361572 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.361737 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.361802 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:14:43.362270 9337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.252@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
I20260619 08:14:43.368603 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.362597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56580)
0619 08:14:43.362668 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.362680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.362717 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.362825 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.362825 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.362932 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.362964 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.362965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.363552 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.363553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.364031 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.364033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.364350 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.364351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.364392 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.364875 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.364880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.366132 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.367779 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.367780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.367780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.367938 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.368127 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.368128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.368128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.368177 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.368376 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.368376 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.368408 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.368466 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.368541 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:14:43.368669 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.362655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.2.119.252:36871)
0619 08:14:43.362706 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.362708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.362715 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:43.362771 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.362771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.362778 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.362823 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.363028 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.363480 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.364098 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.364275 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.364323 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.366249 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.366250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.366250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.366256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.367688 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.368015 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.368016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.368016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.368066 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.368228 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.368229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.368229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.368322 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.368381 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.368527 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.368590 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:43.370353 9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Bootstrap starting.
I20260619 08:14:43.371006 9337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:43.371277 9337 log.cc:824] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:43.372027 9337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: No bootstrap required, opened a new log
I20260619 08:14:43.373505 9337 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } }
I20260619 08:14:43.373646 9337 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:43.373692 9337 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a4dc2bcea244a2909faf5701933ae2, State: Initialized, Role: FOLLOWER
I20260619 08:14:43.373826 9337 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } }
I20260619 08:14:43.374085 9342 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } } }
I20260619 08:14:43.374125 9337 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:43.374171 9342 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:14:43.375840 9353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:14:43.375898 9353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:14:43.378193 9269 master_runner.cc:428] Detected that this master c8a4dc2bcea244a2909faf5701933ae2 is joining an existing cluster
I20260619 08:14:43.378257 9269 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.119.252:36871
I20260619 08:14:43.387495 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.2.119.254:41109)
0619 08:14:43.379365 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.379367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.379649 (+ 282us) server_negotiation.cc:407] Connection header received
0619 08:14:43.379776 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.379777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.379788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.379844 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.380278 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381178 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382587 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.382915 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382961 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.384404 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.384406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.384406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.384414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.386137 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.386525 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.386526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.386527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.386595 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.386911 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.386912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.386912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.387043 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.387128 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.387310 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.387390 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:43.387513 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48336)
0619 08:14:43.379569 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.379582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.379681 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.379847 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.379847 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.380080 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.380177 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.380178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381281 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.381281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.382474 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.382477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.383394 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.383395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.383448 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.383898 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.383903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.384260 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.386244 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.386245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.386245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.386444 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.386685 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.386686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.386686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.386819 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.387119 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.387119 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.387166 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.387279 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.387394 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":87,"thread_start_us":87,"threads_started":1}
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856882, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.254@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
I20260619 08:14:43.404990 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.2.119.252:36871)
0619 08:14:43.379697 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.379699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.380045 (+ 346us) server_negotiation.cc:407] Connection header received
0619 08:14:43.380121 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.380122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.380129 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.380154 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.380538 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381248 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382379 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.382648 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382687 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.401760 (+ 19073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.401761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.401762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.401770 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.403721 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404166 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.404167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.404167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.404217 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404395 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.404395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.404395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.404527 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.404568 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.404761 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.404855 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:14:43.405436 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56590)
0619 08:14:43.380001 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.380008 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.380062 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.380207 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.380208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.380400 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.380460 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.380462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381372 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.381373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.382289 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.382291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.382765 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.382820 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.383348 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.383353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.401669 (+ 18316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.403811 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.403812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.403812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.404084 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.404282 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404282 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.404283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.404332 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.404608 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.404609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.404646 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.405202 (+ 556us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.405331 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":52,"thread_start_us":59,"threads_started":1}
I20260619 08:14:43.405694 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57704)
0619 08:14:43.379819 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.379829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.379902 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.380083 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.380084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.380319 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.380379 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.380381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381429 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.381430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.382484 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.382485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.383048 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.383049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.383131 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.402024 (+ 18893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.402031 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.402528 (+ 497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.404485 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.404486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.404681 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.404881 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.404882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.404932 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.405358 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.405359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.405409 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.405478 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.405592 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":105,"thread_start_us":64,"threads_started":1}
I20260619 08:14:43.405701 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.389710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56602)
0619 08:14:43.389764 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.389775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.389815 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.390080 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.390081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.390244 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.390281 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.390282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.391081 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.391082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.391731 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.391734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.392139 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.392140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.392213 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.392659 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.392665 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.394120 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.404531 (+ 10411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.404533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.404753 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.404982 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.404983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.405040 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.405391 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.405392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.405439 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.405486 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.405596 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:14:43.406368 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.389780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.2.119.252:36871)
0619 08:14:43.389940 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.389942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.389958 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:14:43.389988 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.389988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.389997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.390086 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.390374 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.390986 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.391826 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.392060 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.392141 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.394248 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.394249 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.394250 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.394256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.404424 (+ 10168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404834 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.404835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.404836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.404895 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.405149 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.405149 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.405150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.405282 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.406093 (+ 811us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.406130 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.406269 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260619 08:14:43.406386 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.379578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.2.119.253:39527)
0619 08:14:43.379628 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.379631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.379877 (+ 246us) server_negotiation.cc:407] Connection header received
0619 08:14:43.379967 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.379968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.379979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.380036 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.380469 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.381339 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382587 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.382921 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.382961 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.402646 (+ 19685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.402648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.402649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.402658 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.404407 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404757 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.404758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.404759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.404811 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.404993 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.404994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.404994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.405124 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.405192 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.406227 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.406307 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:14:43.406781 8617 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.119.252:36871: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2"
member_type: NON_VOTER
last_known_addr {
host: "127.2.119.252"
port: 36871
}
attrs {
promote: true
}
}
dest_uuid: "647de3d06aec439e8eda74ec862139c4"
cas_config_opid_index: 5
I20260619 08:14:43.407019 8617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } }
I20260619 08:14:43.407681 9311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:43.407608 9176 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.407716 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8679
I20260619 08:14:43.408576 9176 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.119.252:36871
I20260619 08:14:43.408591 9311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:14:43.408649 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.408715 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.409017 9269 master.cc:537] Master@127.2.119.252:36871 shutting down...
I20260619 08:14:43.415580 9177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 08:14:43.416404 8604 consensus_peers.cc:487] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 -> Peer c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871): Couldn't send request to peer c8a4dc2bcea244a2909faf5701933ae2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:14:43.416410 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:45861
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:41109
--webserver_port=45467
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:14:43.418284 9269 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:43.418408 9269 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:43.418471 9269 tablet_replica.cc:330] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: stopping tablet replica
I20260619 08:14:43.418717 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.407474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57706)
0619 08:14:43.413935 (+ 6461us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.413954 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.414022 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.414309 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.414310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.414312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:43.414367 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.414370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.415477 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.415479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.416918 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.416922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.418154 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.418155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.418234 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.418264 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.418327 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.418646 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6355}
I20260619 08:14:43.418772 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.407627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.2.119.253:39527)
0619 08:14:43.414118 (+ 6491us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.414121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.414130 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:43.414155 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.414156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.414179 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.414241 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:43.414470 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.415366 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.417242 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.418000 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.418095 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.418184 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.418655 (+ 471us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.418698 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6462}
I20260619 08:14:43.418998 9225 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 LEARNER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 08:14:43.419224 9177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 08:14:43.419960 9225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.421331 9255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647de3d06aec439e8eda74ec862139c4. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.421445 9255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:14:43.421511 9255 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: NON_VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: true } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.421613 9255 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:14:43.421736 9177 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: attempting to promote NON_VOTER d1ee07da405c45eb8499bd96215fe128 to VOTER
I20260619 08:14:43.421976 9177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } }
I20260619 08:14:43.422515 9225 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 LEARNER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 08:14:43.422654 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.422539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56608)
0619 08:14:43.422582 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.422623 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 08:14:43.422691 9177 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 08:14:43.422951 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.422876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56612)
0619 08:14:43.422908 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.422929 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:36871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 08:14:43.423034 8604 consensus_peers.cc:487] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 -> Peer c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871): Couldn't send request to peer c8a4dc2bcea244a2909faf5701933ae2. Status: Network error: Client connection negotiation failed: client connection to 127.2.119.252:36871: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 08:14:43.423241 9176 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d1ee07da405c45eb8499bd96215fe128 (127.2.119.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.423996 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.424077 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.424129 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.424180 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.424332 9225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d1ee07da405c45eb8499bd96215fe128 (127.2.119.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.425072 9365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.425175 9365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:14:43.433676 9269 master.cc:552] Master@127.2.119.252:36871 shutdown complete.
I20260619 08:14:43.433748 9269 master_runner.cc:305] Clearing existing system tablet
I20260619 08:14:43.433880 9269 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.433940 9269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.434602 9269 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.435117 9368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.435305 9269 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.435365 9269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.435396 9269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "c8a4dc2bcea244a2909faf5701933ae2"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "cbc15ce5818fa24ef98e55ed8790c6a1"
server_key_iv: "fad0881e0c4a1c90923e6b94dccf04ab"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.436040 9269 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:43.437286 9269 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 08:14:43.437430 9269 log.cc:1196] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 08:14:43.437639 9269 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Deleting consensus metadata
I20260619 08:14:43.437778 9269 master_runner.cc:315] Copying system tablet from 127.2.119.254:41109
I20260619 08:14:43.438464 9269 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: tablet copy: Beginning tablet copy session from remote peer at address 127.2.119.254:41109
I20260619 08:14:43.444914 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.438866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48338 (local address 127.2.119.254:41109)
0619 08:14:43.438915 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.438918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.439049 (+ 131us) server_negotiation.cc:407] Connection header received
0619 08:14:43.439122 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.439123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.439135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.439180 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.439518 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.440034 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.440749 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.440963 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.441047 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.442184 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.442186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.442186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.442194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.443731 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.444088 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.444090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.444090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.444164 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.444378 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.444378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.444379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.444496 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.444581 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.444740 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.444809 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:14:43.444939 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.438805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48338)
0619 08:14:43.438960 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.438978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.439065 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.439196 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.439197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.439383 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.439433 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.439435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.440121 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.440123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.440659 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.440661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.441045 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.441046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.441117 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.441742 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.441747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.442101 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.443825 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.443826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.443826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.444001 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.444216 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.444216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.444217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.444294 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.444571 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.444572 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.444617 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.444702 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.444809 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260619 08:14:43.445503 8637 tablet_copy_service.cc:140] P 647de3d06aec439e8eda74ec862139c4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c8a4dc2bcea244a2909faf5701933ae2 ({username='oryx', principal='oryx/127.2.119.252@KRBTEST.COM'} at 127.0.0.1:48338)
I20260619 08:14:43.445595 8637 tablet_copy_service.cc:161] P 647de3d06aec439e8eda74ec862139c4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c8a4dc2bcea244a2909faf5701933ae2 at {username='oryx', principal='oryx/127.2.119.252@KRBTEST.COM'} at 127.0.0.1:48338: session id = c8a4dc2bcea244a2909faf5701933ae2-00000000000000000000000000000000
I20260619 08:14:43.446050 8637 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:14:43.446512 9269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 08:14:43.447438 9269 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: tablet copy: Starting download of 0 data blocks...
I20260619 08:14:43.447595 9269 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: tablet copy: Starting download of 1 WAL segments...
I20260619 08:14:43.448256 9269 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:14:43.449033 8637 tablet_copy_service.cc:342] P 647de3d06aec439e8eda74ec862139c4: Request end of tablet copy session c8a4dc2bcea244a2909faf5701933ae2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.119.252@KRBTEST.COM'} at 127.0.0.1:48338
I20260619 08:14:43.449111 8637 tablet_copy_service.cc:433] P 647de3d06aec439e8eda74ec862139c4: ending tablet copy session c8a4dc2bcea244a2909faf5701933ae2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c8a4dc2bcea244a2909faf5701933ae2
I20260619 08:14:43.451885 9269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.453109 9378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.453114 9375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.453964 9376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.457492 9269 server_base.cc:1034] running on GCE node
I20260619 08:14:43.457592 9269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.457727 9269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.458846 9269 hybrid_clock.cc:648] HybridClock initialized: now 1781856883458838 us; error 26 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.460278 9269 init.cc:377] Logged in from keytab as oryx/127.2.119.252@KRBTEST.COM (short username oryx)
I20260619 08:14:43.460598 9269 webserver.cc:460] Webserver started at http://127.2.119.252:40657/ using document root <none> and password file <none>
I20260619 08:14:43.460726 9269 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.460760 9269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.461432 9269 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.462007 9385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.462162 9269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.462208 9269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "c8a4dc2bcea244a2909faf5701933ae2"
format_stamp: "Formatted at 2026-06-19 08:14:43 on dist-test-slave-d0b5"
server_key: "cbc15ce5818fa24ef98e55ed8790c6a1"
server_key_iv: "fad0881e0c4a1c90923e6b94dccf04ab"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.462270 9269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.484722 9269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.486598 9269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.489799 9269 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.252:36871
I20260619 08:14:43.489814 9437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.252:36871 every 8 connection(s)
I20260619 08:14:43.490134 9269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
I20260619 08:14:43.490891 9438 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:14:43.491303 9438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Bootstrap starting.
I20260619 08:14:43.493835 9438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:14:43.494056 9438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: Bootstrap complete.
I20260619 08:14:43.494269 9438 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } }
I20260619 08:14:43.494361 9438 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c8a4dc2bcea244a2909faf5701933ae2, State: Initialized, Role: LEARNER
I20260619 08:14:43.494406 9438 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } }
I20260619 08:14:43.494583 9438 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:43.494570 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.494869 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:14:43.496519 9452 catalog_manager.cc:1259] Loaded cluster ID: df75f46532fb4a7da806e75696f4c5cc
I20260619 08:14:43.496618 9452 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: loading cluster ID for follower catalog manager: success
I20260619 08:14:43.497707 9452 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: acquiring CA information for follower catalog manager: success
I20260619 08:14:43.497996 9452 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:14:43.538770 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:43.539079 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:43.539119 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:43.540864 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:43.540942 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:43.540987 9363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:43.542713 9363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:45861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=45467
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:43.542948 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:43.543147 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.548321 9458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.548338 9460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.548338 9457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.549235 9363 server_base.cc:1034] running on GCE node
I20260619 08:14:43.549537 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.549883 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.551034 9363 hybrid_clock.cc:648] HybridClock initialized: now 1781856883551010 us; error 34 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.553853 9363 init.cc:377] Logged in from keytab as oryx/127.2.119.193@KRBTEST.COM (short username oryx)
I20260619 08:14:43.554347 9363 webserver.cc:460] Webserver started at http://127.2.119.193:45467/ using document root <none> and password file <none>
I20260619 08:14:43.554586 9363 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.554643 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.556025 9363 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:43.556641 9467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.556813 9363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:43.556914 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "4b9b52c47e2e4d5082a28ae05789bd6e"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "761285e6d29da8d515218d9846b9f969"
server_key_iv: "54d41e2cdb94b12dca52231f6db39053"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.557255 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.574378 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.576514 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:43.576725 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.577015 9363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:43.577354 9363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:43.577389 9363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.577441 9363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:43.577467 9363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.583314 9363 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:45861
I20260619 08:14:43.583359 9580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:45861 every 8 connection(s)
I20260619 08:14:43.583710 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
I20260619 08:14:43.593762 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9363
I20260619 08:14:43.593916 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8813
I20260619 08:14:43.596184 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:47289 (local address 127.2.119.254:41109)
0619 08:14:43.584554 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.584556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.584768 (+ 212us) server_negotiation.cc:407] Connection header received
0619 08:14:43.586177 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.586179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.586192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.586244 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.586895 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587693 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589331 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589609 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589675 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.592284 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.592285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.592285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.592293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.594172 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595011 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.595012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.595013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.595078 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595372 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.595373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.595374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.595527 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.595570 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.596013 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.596094 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:43.596333 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:47289)
0619 08:14:43.584663 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.584688 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.585826 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.586248 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.586249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.586596 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.586781 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.586787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587767 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.587768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.588812 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.588815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589687 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.589768 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.590085 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.590090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.592193 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.594693 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.594694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.594694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.594916 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.595147 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.595148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.595286 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.595619 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.595623 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.595804 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.595939 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.596101 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":64,"thread_start_us":82,"threads_started":1}
I20260619 08:14:43.596419 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46329 (local address 127.2.119.252:36871)
0619 08:14:43.584596 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.584598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.584763 (+ 165us) server_negotiation.cc:407] Connection header received
0619 08:14:43.585947 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.585949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.585964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.586034 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.586894 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587878 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.588741 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589034 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589099 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.592830 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.592832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.592832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.592841 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.595143 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595535 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.595536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.595537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.595597 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595790 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.595791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.595791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.595929 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.595988 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.596180 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.596273 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260619 08:14:43.596782 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.2.119.193:46329)
0619 08:14:43.584645 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.584686 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.585832 (+ 1146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.586140 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.586144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.586589 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.586781 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.586787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587984 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.587985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.588649 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.588650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589112 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.589208 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.589799 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.589811 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.592249 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.595207 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.595208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.595401 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.595661 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.595662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.595662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.595740 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.596017 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.596018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.596064 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.596493 (+ 429us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.596622 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":8,"thread_start_us":82,"threads_started":1}
I20260619 08:14:43.597191 9581 heartbeater.cc:347] Connected to a master server at 127.2.119.252:36871
I20260619 08:14:43.597285 9581 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.597522 9581 heartbeater.cc:511] Master 127.2.119.252:36871 requested a full tablet report, sending...
I20260619 08:14:43.598026 9402 ts_manager.cc:194] Registered new tserver with Master: 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
I20260619 08:14:43.599422 9582 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:43.599468 9582 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.599519 9582 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:43.599828 8617 ts_manager.cc:194] Re-registered known tserver with Master: 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
I20260619 08:14:43.600214 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:47289
I20260619 08:14:43.601068 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:39185
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:41109
--webserver_port=44555
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:14:43.601655 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:52197 (local address 127.2.119.253:39527)
0619 08:14:43.584998 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.585000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.585008 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:43.586057 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.586058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.586072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.586123 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.586877 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587472 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.588904 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589187 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589246 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.591492 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.591494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.591494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.591502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.593139 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.600855 (+ 7716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.600856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.600856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.600917 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.601153 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.601154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.601154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.601328 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.601382 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.601531 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.601594 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:14:43.601795 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.584764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.2.119.193:52197)
0619 08:14:43.584913 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.584923 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.585837 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.586141 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.586144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.586575 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.586781 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.586785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.587590 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.587591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.588813 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.588815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.589272 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.589273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.589353 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.589932 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.589937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.591396 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.593236 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.593242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.593244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.593533 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.600991 (+ 7458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.600992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.600992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.601066 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.601390 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.601391 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.601439 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.601506 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.601635 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260619 08:14:43.602156 9583 heartbeater.cc:347] Connected to a master server at 127.2.119.253:39527
I20260619 08:14:43.602207 9583 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.602279 9583 heartbeater.cc:511] Master 127.2.119.253:39527 requested a full tablet report, sending...
I20260619 08:14:43.602605 9215 ts_manager.cc:194] Registered new tserver with Master: 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
W20260619 08:14:43.684180 9589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:43.684377 9589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:43.684408 9589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:43.685691 9589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:43.685734 9589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:43.685760 9589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:43.687232 9589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=44555
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:43.687493 9589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:43.687726 9589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.692994 9594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.692977 9595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.693320 9589 server_base.cc:1034] running on GCE node
W20260619 08:14:43.692970 9597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.693580 9589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.693827 9589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.695082 9589 hybrid_clock.cc:648] HybridClock initialized: now 1781856883695067 us; error 31 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.697865 9589 init.cc:377] Logged in from keytab as oryx/127.2.119.194@KRBTEST.COM (short username oryx)
I20260619 08:14:43.698292 9589 webserver.cc:460] Webserver started at http://127.2.119.194:44555/ using document root <none> and password file <none>
I20260619 08:14:43.698549 9589 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.698635 9589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.700057 9589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:43.700765 9604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.701095 9589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:43.701205 9589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "d34658d33b794e7ca29feabb708aba5c"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "3d1f6bbd34e41c2479d401c196076c74"
server_key_iv: "db78678771e8609ddc49ad0252cc82ee"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.701622 9589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.720474 9589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.722693 9589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:43.722859 9589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.723102 9589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:43.723420 9589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:43.723476 9589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.723518 9589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:43.723546 9589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.729290 9589 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:39185
I20260619 08:14:43.729362 9717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:39185 every 8 connection(s)
I20260619 08:14:43.729684 9589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:43.736416 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9589
I20260619 08:14:43.736586 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8947
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
I20260619 08:14:43.742765 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:41103 (local address 127.2.119.253:39527)
0619 08:14:43.730509 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.730512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.730706 (+ 194us) server_negotiation.cc:407] Connection header received
0619 08:14:43.731822 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.731823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.731836 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.731906 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.732770 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733574 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735007 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.735321 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735420 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.738306 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.738308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.738308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.738317 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.740350 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741662 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.741663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.741663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.741721 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.742033 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.742034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.742034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.742160 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.742205 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.742584 (+ 379us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.742668 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:14:43.742777 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:34337 (local address 127.2.119.252:36871)
0619 08:14:43.730441 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.730443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.730895 (+ 452us) server_negotiation.cc:407] Connection header received
0619 08:14:43.731906 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.731908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.731933 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.731992 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.732772 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733565 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.734706 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.734983 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735049 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.738657 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.738659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.738659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.738668 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.741039 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741656 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.741657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.741657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.741721 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.742021 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.742022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.742022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.742160 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.742219 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.742614 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.742696 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:43.742825 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:47823 (local address 127.2.119.254:41109)
0619 08:14:43.730834 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.730836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.730966 (+ 130us) server_negotiation.cc:407] Connection header received
0619 08:14:43.731923 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.731925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.731937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.731973 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.732772 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733426 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.734978 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.735219 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735298 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.738986 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.738987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.738988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.738996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.740755 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741673 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.741675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.741675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.741741 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.742015 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.742016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.742016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.742175 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.742219 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.742588 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.742685 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:14:43.743325 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.2.119.194:34337)
0619 08:14:43.730827 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.730837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.731765 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.732006 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.732007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.732488 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.732657 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.732663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733645 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.733646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.734609 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.734612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.735054 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.735208 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.736137 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.736143 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.738581 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.741117 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.741118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741568 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.741794 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.741795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741944 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.742247 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.742253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.742426 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.742923 (+ 497us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.743049 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":69,"thread_start_us":101,"threads_started":1}
I20260619 08:14:43.743325 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.2.119.194:41103)
0619 08:14:43.730570 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.730616 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.731716 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.731981 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.731985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.732502 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.732657 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.732663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733904 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.733905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.734875 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.734877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.735415 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.735491 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.735873 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.735877 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.738187 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.741139 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.741140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741568 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.741795 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.741795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741944 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.742251 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.742254 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.742426 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.742840 (+ 414us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.743125 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":136,"thread_start_us":99,"threads_started":1}
I20260619 08:14:43.743881 9720 heartbeater.cc:347] Connected to a master server at 127.2.119.253:39527
I20260619 08:14:43.743883 9718 heartbeater.cc:347] Connected to a master server at 127.2.119.252:36871
I20260619 08:14:43.744010 9718 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.744233 9718 heartbeater.cc:511] Master 127.2.119.252:36871 requested a full tablet report, sending...
I20260619 08:14:43.744365 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.730720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:47823)
0619 08:14:43.730927 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.730933 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.731765 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.732051 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.732054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.732439 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.732656 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.732661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.733541 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.733542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.734884 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.734886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.735297 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.735298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.735367 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.735761 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.735769 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.738883 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.740851 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.740915 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.740917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741573 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.741810 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.741811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.741811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.741944 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.742266 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.742267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.742426 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.744044 (+ 1618us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.744170 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":80,"thread_start_us":58,"threads_started":1}
I20260619 08:14:43.744671 9719 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:43.744722 9719 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.744729 9402 ts_manager.cc:194] Registered new tserver with Master: d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
I20260619 08:14:43.744805 9719 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:43.744011 9720 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.745059 9720 heartbeater.cc:511] Master 127.2.119.253:39527 requested a full tablet report, sending...
I20260619 08:14:43.745070 8617 ts_manager.cc:194] Re-registered known tserver with Master: d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
I20260619 08:14:43.745399 9215 ts_manager.cc:194] Registered new tserver with Master: d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
I20260619 08:14:43.745525 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:47823
I20260619 08:14:43.745904 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:33301
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:41109
--webserver_port=35059
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:43.828106 9725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:43.828312 9725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:43.828336 9725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:43.829540 9725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:43.829586 9725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:43.829612 9725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:43.831094 9725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:33301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=35059
--tserver_master_addrs=127.2.119.254:41109,127.2.119.253:39527,127.2.119.252:36871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:43.831349 9725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:43.831610 9725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:43.837109 9730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.837036 9731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:43.837041 9733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:43.837177 9725 server_base.cc:1034] running on GCE node
I20260619 08:14:43.837498 9725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:43.837699 9725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:43.838857 9725 hybrid_clock.cc:648] HybridClock initialized: now 1781856883838842 us; error 29 us; skew 500 ppm
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:43.841670 9725 init.cc:377] Logged in from keytab as oryx/127.2.119.195@KRBTEST.COM (short username oryx)
I20260619 08:14:43.842062 9725 webserver.cc:460] Webserver started at http://127.2.119.195:35059/ using document root <none> and password file <none>
I20260619 08:14:43.842291 9725 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:43.842350 9725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:43.843820 9725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:43.844656 9740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.844858 9725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:43.844951 9725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
format_stamp: "Formatted at 2026-06-19 08:14:42 on dist-test-slave-d0b5"
server_key: "4f2db21736e11d71a32def8238295e4a"
server_key_iv: "a74e5c5b59746bf204b6bbe4737b73f2"
server_key_version: "encryptionkey@0"
I20260619 08:14:43.845350 9725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:43.860543 9725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:43.863353 9725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:43.863539 9725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:43.863801 9725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:43.864145 9725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:43.864200 9725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.864234 9725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:43.864252 9725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:43.865203 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.861730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.2.119.252:36871)
0619 08:14:43.861786 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.861788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.861829 (+ 41us) server_negotiation.cc:407] Connection header received
0619 08:14:43.861924 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.861925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.861949 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.862017 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:43.862157 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.862861 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.864228 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.864785 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.864856 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.864907 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.865111 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.865152 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:43.865204 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.861650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56620)
0619 08:14:43.861759 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.861772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.861861 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.862027 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.862028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.862031 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:43.862087 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.862090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.862981 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.862983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.864112 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.864116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.864889 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.864890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.864973 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.864978 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.865055 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.865112 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 08:14:43.865609 9412 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 08:14:43.865850 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 647de3d06aec439e8eda74ec862139c4. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.865994 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: LEARNER
I20260619 08:14:43.870550 9725 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33301
I20260619 08:14:43.870610 9853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33301 every 8 connection(s)
I20260619 08:14:43.870934 9725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.252@KRBTEST.COM
Jun 19 08:14:43 dist-test-slave-d0b5 krb5kdc[8572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856883, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.253@KRBTEST.COM
I20260619 08:14:43.880795 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9725
I20260619 08:14:43.882174 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.871660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:54751 (local address 127.2.119.254:41109)
0619 08:14:43.871711 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.871714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.871929 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:14:43.873056 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.873057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.873070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.873121 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.873921 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874409 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.875431 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.875640 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.875724 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.878598 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.878600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.878600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.878608 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.880214 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.880937 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.880938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.880938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.881003 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881344 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.881345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.881346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.881505 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.881557 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.881972 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.882067 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:43.882735 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.871640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:60593 (local address 127.2.119.252:36871)
0619 08:14:43.871869 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.871871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.871934 (+ 63us) server_negotiation.cc:407] Connection header received
0619 08:14:43.873095 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.873097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.873110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.873158 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.873931 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874663 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.875802 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.876011 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.876043 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.879499 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.879500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.879501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.879510 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.881116 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881571 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.881573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.881573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.881633 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881875 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.881876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.881876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.882007 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.882070 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.882537 (+ 467us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.882616 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260619 08:14:43.882731 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.871618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:54751)
0619 08:14:43.871802 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.871844 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.872929 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.873190 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.873194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.873619 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.873818 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.873823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874510 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.874510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.875322 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.875324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.875733 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.875734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.875890 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.876591 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.876601 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.878469 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.880492 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.880499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.880502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.880833 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.881100 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.881101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.881228 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.881611 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.881617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.881785 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.882269 (+ 484us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.882458 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":123,"threads_started":1}
I20260619 08:14:43.883368 9855 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41109
I20260619 08:14:43.883463 9855 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.883656 9855 heartbeater.cc:511] Master 127.2.119.254:41109 requested a full tablet report, sending...
I20260619 08:14:43.883698 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.872125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:38423 (local address 127.2.119.253:39527)
0619 08:14:43.872168 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.872170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.872177 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:43.873047 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.873049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.873061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.873093 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.873924 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874691 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.875810 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.876085 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.876151 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.879779 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.879781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.879781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.879789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.882380 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.882744 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.882746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.882746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.882803 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.883026 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.883028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.883028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.883163 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.883226 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.883536 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.883609 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:43.884083 8617 ts_manager.cc:194] Re-registered known tserver with Master: 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301)
I20260619 08:14:43.884056 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.871926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.2.119.195:38423)
0619 08:14:43.872077 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.872087 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.872953 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.873198 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.873198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.873622 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.873819 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.873825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874773 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.874773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.875702 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.875703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.876203 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.876204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.876261 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.876859 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.876865 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.879418 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.882459 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.882460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.882460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.882663 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.882865 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.882866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.882867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.882936 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.883230 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.883231 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.883270 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.883359 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.883459 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":33,"thread_start_us":72,"threads_started":1}
I20260619 08:14:43.884516 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.871620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.2.119.195:60593)
0619 08:14:43.871806 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.871845 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.872957 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.873197 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.873198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.873625 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:43.873820 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.873825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.874762 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.874762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.875700 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.875703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.876102 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.876102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.876228 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.876913 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:43.876917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:43.879425 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:43.881208 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.881209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.881472 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.881720 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:43.881721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:43.881721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:43.881796 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:43.882107 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:43.882108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:43.882153 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.883538 (+ 1385us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.883660 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":76,"thread_start_us":108,"threads_started":1}
I20260619 08:14:43.884603 8617 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:54751
I20260619 08:14:43.884919 9854 heartbeater.cc:347] Connected to a master server at 127.2.119.252:36871
I20260619 08:14:43.884935 9856 heartbeater.cc:347] Connected to a master server at 127.2.119.253:39527
I20260619 08:14:43.884990 9854 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.884991 9856 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:43.885084 9856 heartbeater.cc:511] Master 127.2.119.253:39527 requested a full tablet report, sending...
I20260619 08:14:43.885084 9854 heartbeater.cc:511] Master 127.2.119.252:36871 requested a full tablet report, sending...
I20260619 08:14:43.885454 9402 ts_manager.cc:194] Registered new tserver with Master: 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301)
I20260619 08:14:43.885880 9215 ts_manager.cc:194] Registered new tserver with Master: 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301)
I20260619 08:14:43.892091 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.881889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.2.119.252:36871)
0619 08:14:43.882815 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.882816 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.882823 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:43.882841 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.882842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.882852 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.882905 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.885847 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.886341 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.887501 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.887701 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.887784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.889240 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.889242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.889242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.889251 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.890831 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.891254 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.891255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.891255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.891311 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.891520 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.891522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.891522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.891692 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.891815 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.891909 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.891987 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":57,"threads_started":1}
I20260619 08:14:43.892282 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.882614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.2.119.254:41109)
0619 08:14:43.882650 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.882651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.884162 (+ 1511us) server_negotiation.cc:407] Connection header received
0619 08:14:43.884366 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.884367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.884377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.884404 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.885681 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.886458 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.887432 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.887716 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.887780 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.889479 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.889481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.889481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.889488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.891024 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.891459 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.891460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.891461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.891514 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.891691 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.891692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.891692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.891814 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.891906 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.892111 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.892183 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:14:43.893013 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.883039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.2.119.253:39527)
0619 08:14:43.883770 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.883772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.885945 (+ 2173us) server_negotiation.cc:407] Connection header received
0619 08:14:43.886029 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.886030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.886039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.886066 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:43.886769 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.887218 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.888225 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.888479 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.888539 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.890172 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:43.890173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:43.890174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:43.890181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:43.891856 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.892209 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.892210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.892210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.892262 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:43.892481 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:43.892482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:43.892483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:43.892605 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:43.892706 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.892850 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.892927 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":57,"threads_started":1}
I20260619 08:14:43.894297 8617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:43.896207 8617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:43.896979 9366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: NON_VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: true } } }
I20260619 08:14:43.897101 9366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.898463 9366 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: attempting to promote NON_VOTER c8a4dc2bcea244a2909faf5701933ae2 to VOTER
I20260619 08:14:43.898622 9366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:43.899121 9412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 LEARNER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 08:14:43.899224 9225 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Refusing update from remote peer 647de3d06aec439e8eda74ec862139c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 08:14:43.899320 9871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 08:14:43.899592 9871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 08:14:43.900059 9366 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.900202 9412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.900373 9225 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901086 9366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901181 9366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.901247 9366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901321 9366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.901366 9366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901429 9366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:43.901515 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901610 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:14:43.901702 9365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "647de3d06aec439e8eda74ec862139c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:43.901777 9365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:14:43.906662 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33301 (local address 127.0.0.1:55972)
0619 08:14:43.902603 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.902615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.902652 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.903232 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.903233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.903235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:43.903280 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.903282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904516 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.904517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.905703 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.905705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.906449 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.906450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.906508 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.906511 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.906550 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.906593 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:14:43.906687 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.2.119.195:33301)
0619 08:14:43.903082 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.903085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.903093 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:43.903124 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.903126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.903138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.903226 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:43.903348 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904426 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.905812 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.906361 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.906455 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.906543 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.906579 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.906615 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":98,"threads_started":1}
I20260619 08:14:43.907142 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:39185 (local address 127.0.0.1:48434)
0619 08:14:43.902834 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.902843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.902889 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.903161 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.903163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.903165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:43.903219 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.903221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904588 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.904589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.905539 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.905542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.906873 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.906874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.906962 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.906967 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.907029 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.907069 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":34,"thread_start_us":96,"threads_started":1}
I20260619 08:14:43.907415 9788 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c (DEFAULT_TABLE table=test-table [id=f362f8dc449441dfa1d7e0d5259dbb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:43.907544 9881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.2.119.194:39185)
0619 08:14:43.902911 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.902916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.902927 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:43.902969 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.902972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.902988 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.903089 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:43.903284 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904509 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.905638 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.906392 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.906467 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.907384 (+ 917us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.907414 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.907461 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260619 08:14:43.907792 9788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:43.907810 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:45861 (local address 127.0.0.1:57906)
0619 08:14:43.903320 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:43.903330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:43.903376 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:43.903626 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:43.903627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:43.903629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:43.903664 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.903666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904671 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.904672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.905883 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:43.905885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.907573 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.907574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:43.907641 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.907646 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:43.907700 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:43.907737 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":127,"thread_start_us":80,"threads_started":1}
I20260619 08:14:43.908061 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:43.902856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.2.119.193:45861)
0619 08:14:43.903057 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:43.903061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:43.903433 (+ 372us) server_negotiation.cc:407] Connection header received
0619 08:14:43.903475 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:43.903477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:43.903491 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:43.903561 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:43.903753 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.904583 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.905973 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:43.907388 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:43.907458 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:43.907536 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:43.907941 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 08:14:43.907982 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260619 08:14:43.908156 9652 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c (DEFAULT_TABLE table=test-table [id=f362f8dc449441dfa1d7e0d5259dbb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:43.908514 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:43.908686 9515 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c (DEFAULT_TABLE table=test-table [id=f362f8dc449441dfa1d7e0d5259dbb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:43.908982 9515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:43.911141 9883 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Bootstrap starting.
I20260619 08:14:43.911387 9884 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Bootstrap starting.
I20260619 08:14:43.911787 9883 tablet_bootstrap.cc:654] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:43.911964 9884 tablet_bootstrap.cc:654] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:43.912086 9883 log.cc:824] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:43.912170 9885 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Bootstrap starting.
I20260619 08:14:43.912261 9884 log.cc:824] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:43.912761 9885 tablet_bootstrap.cc:654] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:43.913095 9885 log.cc:824] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:43.913197 9883 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: No bootstrap required, opened a new log
I20260619 08:14:43.913268 9883 ts_tablet_manager.cc:1397] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:43.913415 9884 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: No bootstrap required, opened a new log
I20260619 08:14:43.913478 9884 ts_tablet_manager.cc:1397] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:43.913844 9885 tablet_bootstrap.cc:492] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: No bootstrap required, opened a new log
I20260619 08:14:43.913932 9885 ts_tablet_manager.cc:1397] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:43.915248 9885 raft_consensus.cc:348] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.915406 9885 raft_consensus.cc:374] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:43.915452 9885 raft_consensus.cc:729] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9b52c47e2e4d5082a28ae05789bd6e, State: Initialized, Role: FOLLOWER
I20260619 08:14:43.915365 9883 raft_consensus.cc:348] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.915529 9883 raft_consensus.cc:374] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:43.915562 9885 consensus_queue.cc:260] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.915777 9883 raft_consensus.cc:729] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34658d33b794e7ca29feabb708aba5c, State: Initialized, Role: FOLLOWER
I20260619 08:14:43.915827 9885 ts_tablet_manager.cc:1428] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:14:43.915928 9883 consensus_queue.cc:260] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.915866 9884 raft_consensus.cc:348] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.916011 9884 raft_consensus.cc:374] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:43.916054 9884 raft_consensus.cc:729] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6c7cb9cb6a46a89dd36e44dfccd838, State: Initialized, Role: FOLLOWER
I20260619 08:14:43.916163 9582 heartbeater.cc:503] Master 127.2.119.254:41109 was elected leader, sending a full tablet report...
I20260619 08:14:43.916208 9883 ts_tablet_manager.cc:1428] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:43.916173 9884 consensus_queue.cc:260] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:43.916342 9719 heartbeater.cc:503] Master 127.2.119.254:41109 was elected leader, sending a full tablet report...
I20260619 08:14:43.916409 9884 ts_tablet_manager.cc:1428] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:43.916615 9855 heartbeater.cc:503] Master 127.2.119.254:41109 was elected leader, sending a full tablet report...
W20260619 08:14:43.980465 9721 tablet.cc:2369] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:14:44.084740 9584 tablet.cc:2369] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:44.114825 9890 raft_consensus.cc:482] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:44.114964 9890 raft_consensus.cc:504] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:44.115404 9890 leader_election.cc:288] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301), 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
I20260619 08:14:44.118474 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.115637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33301 (local address 127.2.119.194:43789)
0619 08:14:44.115729 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:44.115744 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:44.115821 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:44.116005 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:44.116006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:44.116008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:44.116091 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:44.116094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.116763 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.116764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:44.117689 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:44.117691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.118228 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.118229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:44.118316 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.118320 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:44.118385 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:14:44.118415 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":1}
I20260619 08:14:44.118626 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.115737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:43789 (local address 127.2.119.195:33301)
0619 08:14:44.115795 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.115798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.115882 (+ 84us) server_negotiation.cc:407] Connection header received
0619 08:14:44.115914 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.115915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.115925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.115996 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:44.116172 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.116679 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.117807 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.118147 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.118204 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.118211 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.118498 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.118560 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:14:44.118673 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.115639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:45861 (local address 127.2.119.194:36829)
0619 08:14:44.115825 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:44.115835 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:44.115898 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:44.116068 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:44.116069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:44.116070 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:44.116151 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:44.116153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.116851 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.116852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:44.117677 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:44.117680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.118288 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.118289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:44.118508 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.118511 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:44.118567 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:14:44.118603 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":43}
I20260619 08:14:44.118834 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.115749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:36829 (local address 127.2.119.193:45861)
0619 08:14:44.115798 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.115799 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.115908 (+ 109us) server_negotiation.cc:407] Connection header received
0619 08:14:44.115984 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.115986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.115997 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.116052 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:44.116232 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.116763 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.117767 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.118201 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.118283 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.118292 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.118697 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.118744 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:14:44.119002 9808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef7cedf7c0a4a10acea25d91c17fb5c" candidate_uuid: "d34658d33b794e7ca29feabb708aba5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" is_pre_election: true
I20260619 08:14:44.119131 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef7cedf7c0a4a10acea25d91c17fb5c" candidate_uuid: "d34658d33b794e7ca29feabb708aba5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" is_pre_election: true
I20260619 08:14:44.119195 9808 raft_consensus.cc:2454] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d34658d33b794e7ca29feabb708aba5c in term 0.
I20260619 08:14:44.119297 9535 raft_consensus.cc:2454] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d34658d33b794e7ca29feabb708aba5c in term 0.
I20260619 08:14:44.119454 9606 leader_election.cc:302] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6c7cb9cb6a46a89dd36e44dfccd838, d34658d33b794e7ca29feabb708aba5c; no voters:
I20260619 08:14:44.119586 9890 raft_consensus.cc:2781] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:44.119639 9890 raft_consensus.cc:482] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:44.119668 9890 raft_consensus.cc:3037] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:44.120321 9890 raft_consensus.cc:504] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:44.120479 9890 leader_election.cc:288] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [CANDIDATE]: Term 1 election: Requested vote from peers 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301), 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861)
I20260619 08:14:44.120595 9808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef7cedf7c0a4a10acea25d91c17fb5c" candidate_uuid: "d34658d33b794e7ca29feabb708aba5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
I20260619 08:14:44.120678 9808 raft_consensus.cc:3037] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:44.120661 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef7cedf7c0a4a10acea25d91c17fb5c" candidate_uuid: "d34658d33b794e7ca29feabb708aba5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e"
I20260619 08:14:44.120772 9535 raft_consensus.cc:3037] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:44.121487 9808 raft_consensus.cc:2454] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d34658d33b794e7ca29feabb708aba5c in term 1.
I20260619 08:14:44.121650 9606 leader_election.cc:302] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6c7cb9cb6a46a89dd36e44dfccd838, d34658d33b794e7ca29feabb708aba5c; no voters:
I20260619 08:14:44.121690 9535 raft_consensus.cc:2454] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d34658d33b794e7ca29feabb708aba5c in term 1.
I20260619 08:14:44.121754 9890 raft_consensus.cc:2781] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:44.121934 9890 raft_consensus.cc:686] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 1 LEADER]: Becoming Leader. State: Replica: d34658d33b794e7ca29feabb708aba5c, State: Running, Role: LEADER
I20260619 08:14:44.122057 9890 consensus_queue.cc:237] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
W20260619 08:14:44.122177 9857 tablet.cc:2369] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:44.122969 8616 catalog_manager.cc:5495] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c reported cstate change: term changed from 0 to 1, leader changed from <none> to d34658d33b794e7ca29feabb708aba5c (127.2.119.194). New cstate: current_term: 1 leader_uuid: "d34658d33b794e7ca29feabb708aba5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } health_report { overall_health: HEALTHY } } }
I20260619 08:14:44.135504 9881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.132974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.2.119.194:39185)
0619 08:14:44.133044 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.133046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.133057 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:44.133114 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.133115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.133132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.133209 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:44.133393 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.133899 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.134674 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.134882 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.134969 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.135164 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:44.135266 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:44.135362 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.135381 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.135410 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:14:44.137512 9808 raft_consensus.cc:1261] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Refusing update from remote peer d34658d33b794e7ca29feabb708aba5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:44.137526 9535 raft_consensus.cc:1261] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Refusing update from remote peer d34658d33b794e7ca29feabb708aba5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:44.137945 9890 consensus_queue.cc:1035] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:44.138137 9892 consensus_queue.cc:1035] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:44.142005 9897 mvcc.cc:205] Tried to move back new op lower bound from 7298485797423947776 to 7298485797364502528. Current Snapshot: MvccSnapshot[applied={T|T < 7298485797423947776}]
I20260619 08:14:44.143857 9896 mvcc.cc:205] Tried to move back new op lower bound from 7298485797423947776 to 7298485797364502528. Current Snapshot: MvccSnapshot[applied={T|T < 7298485797423947776 or (T in {7298485797423947776})}]
I20260619 08:14:44.144171 8617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:44.144277 8617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:44.145783 8617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 647de3d06aec439e8eda74ec862139c4: Sending DeleteTablet for 3 replicas of tablet 7ef7cedf7c0a4a10acea25d91c17fb5c
I20260619 08:14:44.146095 9788 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:44 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:55972
I20260619 08:14:44.146193 9515 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:44 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:57906
I20260619 08:14:44.146421 9905 tablet_replica.cc:330] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: stopping tablet replica
I20260619 08:14:44.146590 9905 raft_consensus.cc:2229] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:44.146812 9905 raft_consensus.cc:2258] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:44.147047 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef7cedf7c0a4a10acea25d91c17fb5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:44 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:48434
I20260619 08:14:44.147293 9907 tablet_replica.cc:330] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: stopping tablet replica
I20260619 08:14:44.147399 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 8585
I20260619 08:14:44.147436 9907 raft_consensus.cc:2229] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:44.147629 9907 raft_consensus.cc:2258] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:44.148193 9907 ts_tablet_manager.cc:1905] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:44.150081 9907 ts_tablet_manager.cc:1918] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:44.150221 9907 log.cc:1196] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/7ef7cedf7c0a4a10acea25d91c17fb5c
I20260619 08:14:44.150614 9907 ts_tablet_manager.cc:1939] T 7ef7cedf7c0a4a10acea25d91c17fb5c P d34658d33b794e7ca29feabb708aba5c: Deleting consensus metadata
I20260619 08:14:44.153439 9904 tablet_replica.cc:330] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: stopping tablet replica
I20260619 08:14:44.153563 9904 raft_consensus.cc:2229] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:44.153815 9904 raft_consensus.cc:2258] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:14:44.155153 9607 connection.cc:535] server connection from 127.0.0.1:48434 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:14:44.155192 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.146697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.2.119.252:36871)
0619 08:14:44.146757 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.146760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.146768 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:44.146788 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.146789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.146798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.146847 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:44.147675 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.148488 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.149738 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.150061 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.150104 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.151841 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:44.151844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:44.151844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:44.151854 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:44.153772 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.154256 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.154258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.154258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.154326 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.154569 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.154571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.154571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.154726 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:44.154853 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.155001 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.155089 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 08:14:44.155896 9201 connection.cc:535] server connection from 127.0.0.1:57706 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:14:44.155983 9905 ts_tablet_manager.cc:1905] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:44.156895 9904 ts_tablet_manager.cc:1905] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:44.158124 9905 ts_tablet_manager.cc:1918] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:44.158308 9905 log.cc:1196] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/7ef7cedf7c0a4a10acea25d91c17fb5c
I20260619 08:14:44.158699 9905 ts_tablet_manager.cc:1939] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 4b9b52c47e2e4d5082a28ae05789bd6e: Deleting consensus metadata
I20260619 08:14:44.158828 9904 ts_tablet_manager.cc:1918] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:44.158919 9904 log.cc:1196] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/7ef7cedf7c0a4a10acea25d91c17fb5c
I20260619 08:14:44.159226 9904 ts_tablet_manager.cc:1939] T 7ef7cedf7c0a4a10acea25d91c17fb5c P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting consensus metadata
W20260619 08:14:44.159207 9471 connection.cc:441] server connection from 127.0.0.1:57906 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57906 (request call id 1) could send its response
W20260619 08:14:44.159601 9741 connection.cc:441] server connection from 127.0.0.1:55972 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55972 (request call id 1) could send its response
I20260619 08:14:44.165935 9585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.157645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56640 (local address 127.2.119.252:36871)
0619 08:14:44.158254 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.158256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.158392 (+ 136us) server_negotiation.cc:407] Connection header received
0619 08:14:44.158499 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.158500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.158510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.158540 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:44.159701 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.160472 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.161383 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.161643 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.161735 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.163167 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:44.163168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:44.163168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:44.163175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:44.164624 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.164983 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.164985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.164985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.165038 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.165327 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.165329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.165329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.165489 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:44.165603 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.165746 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.165819 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583}
I20260619 08:14:44.167819 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:44.158135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.2.119.253:39527)
0619 08:14:44.158176 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:44.158178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:44.159573 (+ 1395us) server_negotiation.cc:407] Connection header received
0619 08:14:44.159656 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:44.159657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:44.159682 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:44.159755 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:44.160765 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.161624 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.162838 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:44.163177 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:44.163219 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:44.164798 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:44.164799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:44.164799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:44.164807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:44.166638 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.167035 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.167036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.167036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.167100 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:44.167282 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:44.167283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:44.167283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:44.167455 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:44.167557 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:44.167633 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 08:14:44.167713 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:14:45.143424 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.143157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:54055)
0619 08:14:45.143297 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.143375 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260619 08:14:45.143671 9582 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:41109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.144395 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.144073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:44013)
0619 08:14:45.144282 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.144349 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":64,"threads_started":1}
W20260619 08:14:45.144618 9719 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:41109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.144657 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.144352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:53747)
0619 08:14:45.144541 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.144608 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":79,"threads_started":1}
W20260619 08:14:45.144836 9855 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:41109 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.646016 9918 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647de3d06aec439e8eda74ec862139c4)
I20260619 08:14:45.646205 9918 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:45.646507 9918 leader_election.cc:288] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), d1ee07da405c45eb8499bd96215fe128 (127.2.119.253:39527)
I20260619 08:14:45.646602 9919 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647de3d06aec439e8eda74ec862139c4)
I20260619 08:14:45.646699 9919 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:45.646924 9919 leader_election.cc:288] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871)
I20260619 08:14:45.646982 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.646643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48852)
0619 08:14:45.646869 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.646928 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":38,"threads_started":1}
W20260619 08:14:45.647125 9388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.647485 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.647259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48854)
0619 08:14:45.647389 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.647444 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":43,"threads_started":1}
I20260619 08:14:45.647512 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.647428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48870)
0619 08:14:45.647458 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.647484 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 08:14:45.647611 9201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
W20260619 08:14:45.647686 9388 leader_election.cc:334] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.648062 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.647964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48886)
0619 08:14:45.648012 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.648038 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:14:45.648137 9201 leader_election.cc:334] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.650482 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.647097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:36871 (local address 127.0.0.1:56650)
0619 08:14:45.647346 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.647355 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:45.647407 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:45.648087 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:45.648088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:45.648090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:45.648138 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:45.648141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.648780 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.648781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:45.649655 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:45.649657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.650213 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.650214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:45.650275 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:45.650278 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:45.650356 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:14:45.650406 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":38,"threads_started":1}
I20260619 08:14:45.650471 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.647227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.2.119.252:36871)
0619 08:14:45.647377 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:45.647380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:45.647393 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:14:45.647489 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:45.647491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:45.647503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:45.648042 (+ 539us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:45.648220 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.648697 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.649742 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.650123 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.650202 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:45.650214 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:45.650363 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 08:14:45.650412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260619 08:14:45.650559 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.646918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.2.119.253:39527)
0619 08:14:45.647173 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:45.647175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:45.647327 (+ 152us) server_negotiation.cc:407] Connection header received
0619 08:14:45.647798 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:45.647801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:45.647814 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:45.647879 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:45.648041 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.648747 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.649805 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.650193 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.650276 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:45.650289 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:45.650432 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 08:14:45.650470 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":57,"threads_started":1}
I20260619 08:14:45.650768 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.646786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:39527 (local address 127.0.0.1:57724)
0619 08:14:45.646926 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.646935 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:45.647012 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:45.647901 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:45.647902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:45.647905 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:45.647968 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:45.647970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.648826 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.648827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:45.649720 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:45.649723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:45.650277 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:45.650278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:45.650347 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:45.650351 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:45.650424 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:14:45.650471 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":52,"threads_started":1}
I20260619 08:14:45.650765 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ee07da405c45eb8499bd96215fe128" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c8a4dc2bcea244a2909faf5701933ae2" is_pre_election: true
I20260619 08:14:45.650871 9412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ee07da405c45eb8499bd96215fe128 in term 1.
I20260619 08:14:45.651086 9202 leader_election.cc:302] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8a4dc2bcea244a2909faf5701933ae2, d1ee07da405c45eb8499bd96215fe128; no voters: 647de3d06aec439e8eda74ec862139c4
I20260619 08:14:45.651113 9225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8a4dc2bcea244a2909faf5701933ae2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d1ee07da405c45eb8499bd96215fe128" is_pre_election: true
I20260619 08:14:45.651206 9919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:14:45.651248 9225 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8a4dc2bcea244a2909faf5701933ae2 in term 1.
I20260619 08:14:45.651289 9919 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647de3d06aec439e8eda74ec862139c4)
I20260619 08:14:45.651324 9919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:14:45.651394 9388 leader_election.cc:302] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8a4dc2bcea244a2909faf5701933ae2, d1ee07da405c45eb8499bd96215fe128; no voters: 647de3d06aec439e8eda74ec862139c4
I20260619 08:14:45.651480 9918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:14:45.651518 9918 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647de3d06aec439e8eda74ec862139c4)
I20260619 08:14:45.651537 9918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:14:45.652083 9919 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:45.652230 9919 leader_election.cc:288] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 election: Requested vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871)
I20260619 08:14:45.652174 9918 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:45.652319 9918 leader_election.cc:288] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 election: Requested vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), d1ee07da405c45eb8499bd96215fe128 (127.2.119.253:39527)
I20260619 08:14:45.652381 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.652296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48894)
0619 08:14:45.652333 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.652358 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:14:45.652415 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ee07da405c45eb8499bd96215fe128" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c8a4dc2bcea244a2909faf5701933ae2"
I20260619 08:14:45.652486 9412 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d1ee07da405c45eb8499bd96215fe128 in current term 2: Already voted for candidate c8a4dc2bcea244a2909faf5701933ae2 in this term.
I20260619 08:14:45.652541 9225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c8a4dc2bcea244a2909faf5701933ae2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d1ee07da405c45eb8499bd96215fe128"
I20260619 08:14:45.652606 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.652406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48904)
0619 08:14:45.652481 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.652509 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 08:14:45.652609 9225 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c8a4dc2bcea244a2909faf5701933ae2 in current term 2: Already voted for candidate d1ee07da405c45eb8499bd96215fe128 in this term.
I20260619 08:14:45.652658 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.652549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48906)
0619 08:14:45.652593 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.652613 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:14:45.652707 9201 leader_election.cc:334] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.652735 9201 leader_election.cc:302] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1ee07da405c45eb8499bd96215fe128; no voters: 647de3d06aec439e8eda74ec862139c4, c8a4dc2bcea244a2909faf5701933ae2
I20260619 08:14:45.652799 9919 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 08:14:45.652998 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:45.652786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:48910)
0619 08:14:45.652958 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:45.652981 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260619 08:14:45.653074 9388 leader_election.cc:334] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:45.653115 9388 leader_election.cc:302] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c8a4dc2bcea244a2909faf5701933ae2; no voters: 647de3d06aec439e8eda74ec862139c4, d1ee07da405c45eb8499bd96215fe128
I20260619 08:14:45.653178 9918 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 08:14:46.144258 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:46.143999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:42269)
0619 08:14:46.144165 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:46.144209 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260619 08:14:46.145139 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:46.144919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:58449)
0619 08:14:46.145058 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:46.145094 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260619 08:14:46.145399 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:46.145137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:34003)
0619 08:14:46.145322 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:46.145360 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260619 08:14:47.145134 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.144805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:35349)
0619 08:14:47.145031 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.145078 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":168,"threads_started":1}
I20260619 08:14:47.145808 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.145611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:38385)
0619 08:14:47.145717 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.145761 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260619 08:14:47.145992 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.145766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:44941)
0619 08:14:47.145920 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.145952 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":45,"threads_started":1}
I20260619 08:14:47.470717 9934 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:47.470839 9934 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:47.471071 9934 leader_election.cc:288] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871)
I20260619 08:14:47.471328 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ee07da405c45eb8499bd96215fe128" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c8a4dc2bcea244a2909faf5701933ae2" is_pre_election: true
I20260619 08:14:47.471411 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.471191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49180)
0619 08:14:47.471321 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.471357 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":50,"threads_started":1}
I20260619 08:14:47.471451 9412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ee07da405c45eb8499bd96215fe128 in term 2.
I20260619 08:14:47.471632 9202 leader_election.cc:302] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8a4dc2bcea244a2909faf5701933ae2, d1ee07da405c45eb8499bd96215fe128; no voters:
I20260619 08:14:47.471745 9934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 08:14:47.471808 9934 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:47.471836 9934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:14:47.471969 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.471857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49194)
0619 08:14:47.471902 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.471946 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:14:47.472081 9201 leader_election.cc:334] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:47.472975 9934 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:47.473174 9934 leader_election.cc:288] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 election: Requested vote from peers 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109), c8a4dc2bcea244a2909faf5701933ae2 (127.2.119.252:36871)
I20260619 08:14:47.473366 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.473255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49196)
0619 08:14:47.473316 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.473340 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:14:47.473353 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ee07da405c45eb8499bd96215fe128" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c8a4dc2bcea244a2909faf5701933ae2"
I20260619 08:14:47.473448 9412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:14:47.473651 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.473583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49210)
0619 08:14:47.473610 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.473633 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 08:14:47.473759 9201 leader_election.cc:334] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
I20260619 08:14:47.474452 9412 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ee07da405c45eb8499bd96215fe128 in term 3.
I20260619 08:14:47.474650 9202 leader_election.cc:302] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c8a4dc2bcea244a2909faf5701933ae2, d1ee07da405c45eb8499bd96215fe128; no voters: 647de3d06aec439e8eda74ec862139c4
I20260619 08:14:47.474781 9934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:14:47.474949 9934 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [term 3 LEADER]: Becoming Leader. State: Replica: d1ee07da405c45eb8499bd96215fe128, State: Running, Role: LEADER
I20260619 08:14:47.475041 9934 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } }
I20260619 08:14:47.475517 9937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1ee07da405c45eb8499bd96215fe128. Latest consensus state: current_term: 3 leader_uuid: "d1ee07da405c45eb8499bd96215fe128" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:47.475680 9937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:47.475908 9939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:47.476416 9939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:47.476495 9939 catalog_manager.cc:1259] Loaded cluster ID: df75f46532fb4a7da806e75696f4c5cc
I20260619 08:14:47.476536 9939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:47.476758 9939 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:47.476866 9939 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Loaded TSK: 0
I20260619 08:14:47.477029 9939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:14:47.498291 9214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:47.499964 9214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:47.501026 9412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [term 3 FOLLOWER]: Refusing update from remote peer d1ee07da405c45eb8499bd96215fe128: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 08:14:47.501048 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.500918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49226)
0619 08:14:47.500995 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.501023 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 08:14:47.501411 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.501266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.0.0.1:49236)
0619 08:14:47.501330 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.501354 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 08:14:47.501349 9937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 08:14:47.501489 9201 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 -> Peer 647de3d06aec439e8eda74ec862139c4 (127.2.119.254:41109): Couldn't send request to peer 647de3d06aec439e8eda74ec862139c4. Status: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 08:14:47.501936 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1ee07da405c45eb8499bd96215fe128. Latest consensus state: current_term: 3 leader_uuid: "d1ee07da405c45eb8499bd96215fe128" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:47.502075 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:14:47.502496 9934 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d1ee07da405c45eb8499bd96215fe128" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:47.502617 9934 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:47.502835 9941 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d1ee07da405c45eb8499bd96215fe128" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "647de3d06aec439e8eda74ec862139c4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41109 } } peers { permanent_uuid: "d1ee07da405c45eb8499bd96215fe128" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 39527 } attrs { promote: false } } peers { permanent_uuid: "c8a4dc2bcea244a2909faf5701933ae2" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 36871 } attrs { promote: false } } }
I20260619 08:14:47.502955 9941 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a4dc2bcea244a2909faf5701933ae2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:14:47.507946 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:45861 (local address 127.0.0.1:57916)
0619 08:14:47.504472 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.504483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.504570 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.504861 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:47.504862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:47.504863 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:47.504911 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.504913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.505725 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.505726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.506801 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.506803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507714 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.507776 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.507780 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:47.507834 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:47.507877 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260619 08:14:47.507948 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57916 (local address 127.2.119.193:45861)
0619 08:14:47.504723 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.504725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.504733 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:47.504759 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.504760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.504771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.504976 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:47.505082 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.505641 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507267 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507632 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507716 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.507728 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.507844 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.507877 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":54,"threads_started":1}
I20260619 08:14:47.508155 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33301 (local address 127.0.0.1:55976)
0619 08:14:47.504424 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.504436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.504496 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.504789 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:47.504790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:47.504792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:47.504854 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.504856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.506061 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.506062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.507079 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.507081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507917 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.507989 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.507992 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:47.508042 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 08:14:47.508084 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":36}
I20260619 08:14:47.508330 9515 tablet_service.cc:1467] Processing CreateTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f (DEFAULT_TABLE table=test-table [id=8fd6fe697722495db01ea472b5e98cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:47.508401 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.2.119.195:33301)
0619 08:14:47.504561 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.504563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.504571 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:47.504597 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.504598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.504608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.504708 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:47.505083 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.505934 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507165 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507793 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507869 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.507880 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.508250 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.508300 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
I20260619 08:14:47.508466 9515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b387eab7694cfeadcb8f5e9d333f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:47.508503 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:39185 (local address 127.0.0.1:48458)
0619 08:14:47.504652 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.504661 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.504698 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.504919 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:47.504920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:47.504921 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:47.504961 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.504962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.506078 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.506079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.507192 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.507194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507989 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.508322 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.508327 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:47.508386 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:14:47.508429 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":30,"thread_start_us":64,"threads_started":1}
I20260619 08:14:47.508538 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.504593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.2.119.194:39185)
0619 08:14:47.504722 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.504724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.504733 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:47.504793 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.504794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.504805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.504869 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:47.505045 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.505938 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507270 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.507885 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.507950 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.507960 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.508406 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.508461 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260619 08:14:47.509486 9788 tablet_service.cc:1467] Processing CreateTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f (DEFAULT_TABLE table=test-table [id=8fd6fe697722495db01ea472b5e98cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:47.509477 9652 tablet_service.cc:1467] Processing CreateTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f (DEFAULT_TABLE table=test-table [id=8fd6fe697722495db01ea472b5e98cf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:47.509599 9788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b387eab7694cfeadcb8f5e9d333f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:47.509617 9652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74b387eab7694cfeadcb8f5e9d333f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:47.509771 9952 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e: Bootstrap starting.
I20260619 08:14:47.510087 9952 tablet_bootstrap.cc:654] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:47.510878 9952 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e: No bootstrap required, opened a new log
I20260619 08:14:47.510912 9952 ts_tablet_manager.cc:1397] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:47.511093 9954 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Bootstrap starting.
I20260619 08:14:47.511044 9952 raft_consensus.cc:348] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.511193 9952 raft_consensus.cc:374] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:47.511221 9952 raft_consensus.cc:729] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9b52c47e2e4d5082a28ae05789bd6e, State: Initialized, Role: FOLLOWER
I20260619 08:14:47.511277 9952 consensus_queue.cc:260] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.511443 9954 tablet_bootstrap.cc:654] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:47.511744 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.511641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:54899)
0619 08:14:47.511683 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.511714 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:14:47.511443 9952 ts_tablet_manager.cc:1428] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:47.512019 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.511933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:57751)
0619 08:14:47.511970 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.511996 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:14:47.512141 9954 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: No bootstrap required, opened a new log
I20260619 08:14:47.512193 9954 ts_tablet_manager.cc:1397] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:47.512342 9583 heartbeater.cc:503] Master 127.2.119.253:39527 was elected leader, sending a full tablet report...
I20260619 08:14:47.512321 9954 raft_consensus.cc:348] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.512398 9954 raft_consensus.cc:374] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:47.512434 9954 raft_consensus.cc:729] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6c7cb9cb6a46a89dd36e44dfccd838, State: Initialized, Role: FOLLOWER
I20260619 08:14:47.512483 9954 consensus_queue.cc:260] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.512661 9954 ts_tablet_manager.cc:1428] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:47.512773 9955 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: Bootstrap starting.
I20260619 08:14:47.513108 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.513001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:51035)
0619 08:14:47.513042 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.513078 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 08:14:47.513183 9955 tablet_bootstrap.cc:654] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:47.513337 9856 heartbeater.cc:503] Master 127.2.119.253:39527 was elected leader, sending a full tablet report...
I20260619 08:14:47.513579 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.513430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:51077)
0619 08:14:47.513520 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.513545 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 08:14:47.513815 9955 tablet_bootstrap.cc:492] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: No bootstrap required, opened a new log
I20260619 08:14:47.513883 9955 ts_tablet_manager.cc:1397] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:47.514029 9955 raft_consensus.cc:348] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.514106 9955 raft_consensus.cc:374] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:47.514120 9955 raft_consensus.cc:729] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34658d33b794e7ca29feabb708aba5c, State: Initialized, Role: FOLLOWER
I20260619 08:14:47.514182 9955 consensus_queue.cc:260] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.514357 9955 ts_tablet_manager.cc:1428] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:47.514657 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.514565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:36937)
0619 08:14:47.514602 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.514628 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 08:14:47.514812 9720 heartbeater.cc:503] Master 127.2.119.253:39527 was elected leader, sending a full tablet report...
I20260619 08:14:47.515316 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.515088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:42689)
0619 08:14:47.515249 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.515286 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260619 08:14:47.573411 9956 raft_consensus.cc:482] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:47.573511 9956 raft_consensus.cc:504] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.573890 9956 leader_election.cc:288] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301), d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
I20260619 08:14:47.577271 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.574161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33301 (local address 127.2.119.193:40239)
0619 08:14:47.574226 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.574239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.574296 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.574478 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:47.574479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:47.574482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:47.574570 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.574573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.575405 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.575406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.576373 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.576376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.576952 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.576953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.577096 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.577101 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:47.577150 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 08:14:47.577192 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:14:47.577322 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.574243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:40239 (local address 127.2.119.195:33301)
0619 08:14:47.574309 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.574312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.574322 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:47.574356 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.574357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.574367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.574463 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:47.574708 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.575317 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.576458 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.576876 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.576930 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.576937 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.577168 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.577206 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:14:47.577533 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.574294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:39185 (local address 127.2.119.193:45447)
0619 08:14:47.574487 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.574499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.574576 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.574735 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:47.574736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:47.574738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:47.574800 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.574802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.575457 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.575457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.576496 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:47.576499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.577146 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.577148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:47.577216 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.577220 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:47.577395 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 08:14:47.577444 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260619 08:14:47.577636 9808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b387eab7694cfeadcb8f5e9d333f7f" candidate_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" is_pre_election: true
I20260619 08:14:47.577741 9808 raft_consensus.cc:2454] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9b52c47e2e4d5082a28ae05789bd6e in term 0.
I20260619 08:14:47.577844 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.574409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:45447 (local address 127.2.119.194:39185)
0619 08:14:47.574493 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.574495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.574564 (+ 69us) server_negotiation.cc:407] Connection header received
0619 08:14:47.574644 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.574646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.574657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.574718 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:47.574887 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.575342 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.576581 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.577020 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.577105 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.577115 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.577717 (+ 602us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.577765 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:14:47.577930 9469 leader_election.cc:302] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6c7cb9cb6a46a89dd36e44dfccd838, 4b9b52c47e2e4d5082a28ae05789bd6e; no voters:
I20260619 08:14:47.578081 9956 raft_consensus.cc:2781] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:47.578128 9956 raft_consensus.cc:482] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:47.578156 9956 raft_consensus.cc:3037] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:47.578249 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b387eab7694cfeadcb8f5e9d333f7f" candidate_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34658d33b794e7ca29feabb708aba5c" is_pre_election: true
I20260619 08:14:47.578353 9672 raft_consensus.cc:2454] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b9b52c47e2e4d5082a28ae05789bd6e in term 0.
I20260619 08:14:47.578931 9956 raft_consensus.cc:504] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.579074 9956 leader_election.cc:288] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [CANDIDATE]: Term 1 election: Requested vote from peers 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301), d34658d33b794e7ca29feabb708aba5c (127.2.119.194:39185)
I20260619 08:14:47.579246 9808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b387eab7694cfeadcb8f5e9d333f7f" candidate_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838"
I20260619 08:14:47.579321 9808 raft_consensus.cc:3037] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:47.579277 9672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74b387eab7694cfeadcb8f5e9d333f7f" candidate_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34658d33b794e7ca29feabb708aba5c"
I20260619 08:14:47.579372 9672 raft_consensus.cc:3037] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:47.579892 9808 raft_consensus.cc:2454] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9b52c47e2e4d5082a28ae05789bd6e in term 1.
I20260619 08:14:47.580050 9469 leader_election.cc:302] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d6c7cb9cb6a46a89dd36e44dfccd838, 4b9b52c47e2e4d5082a28ae05789bd6e; no voters:
I20260619 08:14:47.580111 9672 raft_consensus.cc:2454] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b9b52c47e2e4d5082a28ae05789bd6e in term 1.
I20260619 08:14:47.580153 9956 raft_consensus.cc:2781] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:47.580304 9956 raft_consensus.cc:686] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [term 1 LEADER]: Becoming Leader. State: Replica: 4b9b52c47e2e4d5082a28ae05789bd6e, State: Running, Role: LEADER
I20260619 08:14:47.580397 9956 consensus_queue.cc:237] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } }
I20260619 08:14:47.580731 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.580510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:56095)
0619 08:14:47.580662 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.580699 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260619 08:14:47.581228 9214 catalog_manager.cc:5495] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193). New cstate: current_term: 1 leader_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b9b52c47e2e4d5082a28ae05789bd6e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:47.600734 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.598340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.2.119.193:45861)
0619 08:14:47.598403 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:47.598405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:47.598413 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:47.598466 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:47.598467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:47.598484 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:47.598542 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:47.598688 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.599157 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.599955 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:47.600172 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:47.600230 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:47.600382 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:47.600478 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:47.600533 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:47.600597 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:14:47.600616 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:14:47.602092 9808 raft_consensus.cc:1261] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Refusing update from remote peer 4b9b52c47e2e4d5082a28ae05789bd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:47.602144 9672 raft_consensus.cc:1261] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Refusing update from remote peer 4b9b52c47e2e4d5082a28ae05789bd6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:47.602348 9962 consensus_queue.cc:1035] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6c7cb9cb6a46a89dd36e44dfccd838" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:47.602473 9956 consensus_queue.cc:1035] T 74b387eab7694cfeadcb8f5e9d333f7f P 4b9b52c47e2e4d5082a28ae05789bd6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d34658d33b794e7ca29feabb708aba5c" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:47.603216 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.603102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:42731)
0619 08:14:47.603138 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.603184 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 08:14:47.603900 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.603544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:34835)
0619 08:14:47.603836 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.603869 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272}
I20260619 08:14:47.604691 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.604498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.195:51715)
0619 08:14:47.604632 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.604664 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260619 08:14:47.605448 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.604446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:35723)
0619 08:14:47.605265 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.605297 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":800}
I20260619 08:14:47.605996 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.605885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.194:35311)
0619 08:14:47.605944 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.605969 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:14:47.606330 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.606222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41109 (local address 127.2.119.193:52681)
0619 08:14:47.606271 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.606305 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:41109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:14:47.606572 9215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:47.606691 9215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:47.608026 9215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1ee07da405c45eb8499bd96215fe128: Sending DeleteTablet for 3 replicas of tablet 74b387eab7694cfeadcb8f5e9d333f7f
I20260619 08:14:47.608363 9515 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:47 UTC) from {username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:57916
I20260619 08:14:47.608409 9788 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:47 UTC) from {username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:55976
I20260619 08:14:47.608438 9652 tablet_service.cc:1514] Processing DeleteTablet for tablet 74b387eab7694cfeadcb8f5e9d333f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:47 UTC) from {username='oryx', principal='oryx/127.2.119.253@KRBTEST.COM'} at 127.0.0.1:48458
I20260619 08:14:47.608604 9975 tablet_replica.cc:330] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: stopping tablet replica
I20260619 08:14:47.608662 9975 raft_consensus.cc:2229] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:47.608683 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9363
I20260619 08:14:47.608704 9975 raft_consensus.cc:2258] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:47.614250 9976 tablet_replica.cc:330] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: stopping tablet replica
I20260619 08:14:47.614344 9976 raft_consensus.cc:2229] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:47.614408 9976 raft_consensus.cc:2258] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:14:47.614456 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:47.613574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:45861 (local address 127.0.0.1:57938)
0619 08:14:47.613641 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.613652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:47.613701 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:47.614409 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:45861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 08:14:47.614517 9975 ts_tablet_manager.cc:1905] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:14:47.614584 9202 catalog_manager.cc:4584] TS 4b9b52c47e2e4d5082a28ae05789bd6e (127.2.119.193:45861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74b387eab7694cfeadcb8f5e9d333f7f: Network error: Client connection negotiation failed: client connection to 127.2.119.193:45861: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:14:47.614732 9976 ts_tablet_manager.cc:1905] T 74b387eab7694cfeadcb8f5e9d333f7f P d34658d33b794e7ca29feabb708aba5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:47.614825 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9589
I20260619 08:14:47.616700 9975 ts_tablet_manager.cc:1918] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:47.616779 9975 log.cc:1196] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/74b387eab7694cfeadcb8f5e9d333f7f
I20260619 08:14:47.616993 9975 ts_tablet_manager.cc:1939] T 74b387eab7694cfeadcb8f5e9d333f7f P 0d6c7cb9cb6a46a89dd36e44dfccd838: Deleting consensus metadata
I20260619 08:14:47.617367 9200 catalog_manager.cc:4841] TS 0d6c7cb9cb6a46a89dd36e44dfccd838 (127.2.119.195:33301): tablet 74b387eab7694cfeadcb8f5e9d333f7f (table test-table [id=8fd6fe697722495db01ea472b5e98cf9]) successfully deleted
I20260619 08:14:47.621696 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:47.621557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:39185 (local address 127.0.0.1:48462)
0619 08:14:47.621625 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:47.621657 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 08:14:47.621879 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9725
I20260619 08:14:47.626861 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9081
I20260619 08:14:47.631433 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9269
2026-06-19T08:14:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6662 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:47 dist-test-slave-d0b5 krb5kdc[9979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:47 dist-test-slave-d0b5 krb5kdc[9979](info): set up 2 sockets
Jun 19 08:14:47 dist-test-slave-d0b5 krb5kdc[9979](info): commencing operation
krb5kdc: starting...
W20260619 08:14:48.674961 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:48 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab.
Entry for principal oryx/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab.
I20260619 08:14:48.802297 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41783
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab
--principal=oryx/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:48.883906 9992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:48.884122 9992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:48.884145 9992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:48.885352 9992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:48.885392 9992 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 08:14:48.885406 9992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:48.885421 9992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:48.885432 9992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:48.885479 9992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:48.887068 9992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41783
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.254.keytab
--principal=oryx/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:48.887336 9992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:48.887565 9992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:14:48.892658 9992 server_base.cc:1034] running on GCE node
W20260619 08:14:48.892632 9997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:48.892668 9998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:48.892632 10000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:48.893153 9992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:48.893455 9992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:48.894595 9992 hybrid_clock.cc:648] HybridClock initialized: now 1781856888894573 us; error 50 us; skew 500 ppm
Jun 19 08:14:48 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856888, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:48.897140 9992 init.cc:377] Logged in from keytab as oryx/127.2.119.254@KRBTEST.COM (short username oryx)
I20260619 08:14:48.897683 9992 webserver.cc:460] Webserver started at http://127.2.119.254:42777/ using document root <none> and password file <none>
I20260619 08:14:48.897913 9992 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:48.897981 9992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:48.898180 9992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:48.899245 9992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "105de698bbb44bfdb8934961e77ffd76"
format_stamp: "Formatted at 2026-06-19 08:14:48 on dist-test-slave-d0b5"
server_key: "ee4806b57bd0d050cf86e05d028a6ed6"
server_key_iv: "204637cfce5743439cfdc25bd42c6bdc"
server_key_version: "encryptionkey@0"
I20260619 08:14:48.899616 9992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "105de698bbb44bfdb8934961e77ffd76"
format_stamp: "Formatted at 2026-06-19 08:14:48 on dist-test-slave-d0b5"
server_key: "ee4806b57bd0d050cf86e05d028a6ed6"
server_key_iv: "204637cfce5743439cfdc25bd42c6bdc"
server_key_version: "encryptionkey@0"
I20260619 08:14:48.900954 9992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:14:48.901829 10007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:48.902009 9992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:48.902156 9992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "105de698bbb44bfdb8934961e77ffd76"
format_stamp: "Formatted at 2026-06-19 08:14:48 on dist-test-slave-d0b5"
server_key: "ee4806b57bd0d050cf86e05d028a6ed6"
server_key_iv: "204637cfce5743439cfdc25bd42c6bdc"
server_key_version: "encryptionkey@0"
I20260619 08:14:48.902233 9992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:48.920825 9992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:48.923128 9992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:48.923301 9992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:48.928231 9992 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41783
I20260619 08:14:48.928284 10069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41783 every 8 connection(s)
I20260619 08:14:48.928618 9992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:48.929265 10070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:48.931510 10070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Bootstrap starting.
I20260619 08:14:48.932055 10070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:48.932291 10070 log.cc:824] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:48.932924 10070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: No bootstrap required, opened a new log
I20260619 08:14:48.935009 10070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } }
I20260619 08:14:48.935144 10070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:48.935168 10070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 105de698bbb44bfdb8934961e77ffd76, State: Initialized, Role: FOLLOWER
I20260619 08:14:48.935304 10070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } }
I20260619 08:14:48.935380 10070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:48.935403 10070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:48.935436 10070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:48.935911 10070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } }
I20260619 08:14:48.935992 10070 leader_election.cc:302] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 105de698bbb44bfdb8934961e77ffd76; no voters:
I20260619 08:14:48.936120 10070 leader_election.cc:288] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:48.936200 10073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:48.936374 10073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [term 1 LEADER]: Becoming Leader. State: Replica: 105de698bbb44bfdb8934961e77ffd76, State: Running, Role: LEADER
I20260619 08:14:48.936386 10070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:48.936519 10073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } }
I20260619 08:14:48.937013 10074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "105de698bbb44bfdb8934961e77ffd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } } }
I20260619 08:14:48.937029 10075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 105de698bbb44bfdb8934961e77ffd76. Latest consensus state: current_term: 1 leader_uuid: "105de698bbb44bfdb8934961e77ffd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "105de698bbb44bfdb8934961e77ffd76" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41783 } } }
I20260619 08:14:48.937196 10075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:48.937398 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 9992
I20260619 08:14:48.937434 10074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:48.937494 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:48.937559 10082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:48.937701 2527 external_mini_cluster.cc:1428] Setting key c4622c9f51fafa7ae5acca7728a044fc
I20260619 08:14:48.938125 10082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:48.940125 10082 catalog_manager.cc:1347] Generated new cluster ID: 273ffe43530744af924cebc759d71e32
I20260619 08:14:48.940183 10082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:14:48 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:48.950930 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:48.938381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.2.119.254:41783)
0619 08:14:48.938675 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:48.938685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:48.938967 (+ 282us) server_negotiation.cc:407] Connection header received
0619 08:14:48.939060 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:48.939066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:48.939110 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:48.939249 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:48.940484 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:48.941971 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:48.943026 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:48.944086 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:48.944203 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:48.947010 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:48.947014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:48.947015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:48.947027 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:48.949030 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:48.949478 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:48.949481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:48.949481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:48.949595 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:48.949803 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:48.949804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:48.949804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:48.950129 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:48.950269 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:48.950528 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 08:14:48.950682 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":73,"threads_started":1}
I20260619 08:14:48.969266 10082 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:48.969944 10082 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:48.979972 10082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Generated new TSK 0
I20260619 08:14:48.980185 10082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
I20260619 08:14:49.013067 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41783
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:49.094233 10096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:49.094427 10096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:49.094452 10096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:49.095552 10096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:49.095588 10096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:49.095608 10096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:49.095625 10096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:49.097051 10096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:49.097226 10096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:49.097482 10096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:49.102401 10104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:49.102438 10102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.102406 10096 server_base.cc:1034] running on GCE node
W20260619 08:14:49.102423 10101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.102789 10096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:49.102978 10096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:49.104164 10096 hybrid_clock.cc:648] HybridClock initialized: now 1781856889104176 us; error 52 us; skew 500 ppm
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:49.106751 10096 init.cc:377] Logged in from keytab as oryx/127.2.119.193@KRBTEST.COM (short username oryx)
I20260619 08:14:49.107175 10096 webserver.cc:460] Webserver started at http://127.2.119.193:35935/ using document root <none> and password file <none>
I20260619 08:14:49.107443 10096 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:49.107512 10096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:49.107664 10096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:49.108697 10096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "a437dc1556804ee5ad20510dd640f8b3"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "51b13cf196bea8fe578937757a446e42"
server_key_iv: "e61b3ec5dd87e1d46fa0c48eae524439"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.109059 10096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "a437dc1556804ee5ad20510dd640f8b3"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "51b13cf196bea8fe578937757a446e42"
server_key_iv: "e61b3ec5dd87e1d46fa0c48eae524439"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.110498 10096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:14:49.111315 10111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.111506 10096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.111613 10096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "a437dc1556804ee5ad20510dd640f8b3"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "51b13cf196bea8fe578937757a446e42"
server_key_iv: "e61b3ec5dd87e1d46fa0c48eae524439"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.111701 10096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:49.135141 10096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:49.137847 10096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:49.138031 10096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:49.138561 10096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:49.138621 10096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.138672 10096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:49.138702 10096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:49.145939 10096 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:39487
I20260619 08:14:49.146023 10226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:39487 every 8 connection(s)
I20260619 08:14:49.146322 10096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:49.148236 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10096
I20260619 08:14:49.148350 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:49.148561 2527 external_mini_cluster.cc:1428] Setting key 7b9b16dbbc9482d47da31d5f506e4468
I20260619 08:14:49.149325 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.139170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:60879 (local address 127.2.119.254:41783)
0619 08:14:49.139225 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.139227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.139479 (+ 252us) server_negotiation.cc:407] Connection header received
0619 08:14:49.140064 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.140066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.140082 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.140160 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.141141 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.141736 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.142799 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.143161 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.143237 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.146075 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.146076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.146077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.146089 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.147657 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.148098 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.148100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.148100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.148166 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.148536 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.148538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.148538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.148729 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.148821 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.149115 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.149212 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:49.149489 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.139161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.193:60879)
0619 08:14:49.139342 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.139379 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.139909 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.140282 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.140287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.140720 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.141022 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.141047 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.141853 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.141854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.142697 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.142698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.143260 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.143261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.143417 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.143984 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.143995 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.145955 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.147752 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.147758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.147760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.147988 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.148236 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.148237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.148237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.148476 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.148810 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.148817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.148967 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.149074 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.149237 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260619 08:14:49.150838 10227 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41783
I20260619 08:14:49.150954 10227 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:49.151142 10227 heartbeater.cc:511] Master 127.2.119.254:41783 requested a full tablet report, sending...
I20260619 08:14:49.151680 10023 ts_manager.cc:194] Registered new tserver with Master: a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487)
I20260619 08:14:49.152602 10023 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:60879
I20260619 08:14:49.157593 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.151027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33677 (local address 127.2.119.254:41783)
0619 08:14:49.151072 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.151075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.151228 (+ 153us) server_negotiation.cc:407] Connection header received
0619 08:14:49.151308 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.151309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.151320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.151386 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.151862 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.152501 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.153612 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.153886 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.153960 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.154981 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.154982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.154983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.154990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.156394 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.156817 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.156818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.156819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.156887 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.157108 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.157108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.157109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.157214 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.157266 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.157458 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.157529 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:49.157680 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.151026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.193:33677)
0619 08:14:49.151161 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.151172 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.151252 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.151445 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.151446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.151667 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.151732 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.151735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.152596 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.152597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.153519 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.153521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.153980 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.153981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.154061 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.154462 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.154467 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.154882 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.156480 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.156481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.156481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.156723 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.156974 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.156975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.156975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.157042 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.157280 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.157281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.157360 (+ 79us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.157437 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.157547 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for oryx/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
I20260619 08:14:49.177515 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41783
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:49.258065 10236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:49.258296 10236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:49.258333 10236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:49.259477 10236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:49.259555 10236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:49.259591 10236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:49.259635 10236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:49.261124 10236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:49.261409 10236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:49.261638 10236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:49.266527 10244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.266781 10236 server_base.cc:1034] running on GCE node
W20260619 08:14:49.266718 10242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:49.266618 10241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.267043 10236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:49.267297 10236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:49.268441 10236 hybrid_clock.cc:648] HybridClock initialized: now 1781856889268413 us; error 44 us; skew 500 ppm
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:49.270812 10236 init.cc:377] Logged in from keytab as oryx/127.2.119.194@KRBTEST.COM (short username oryx)
I20260619 08:14:49.271271 10236 webserver.cc:460] Webserver started at http://127.2.119.194:42261/ using document root <none> and password file <none>
I20260619 08:14:49.271505 10236 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:49.271552 10236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:49.271709 10236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:49.272919 10236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "9914f2a9e5c19170a047501f26b989c5"
server_key_iv: "a8c7892ee80ddfd85e05b7168c50dcb9"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.273377 10236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "9914f2a9e5c19170a047501f26b989c5"
server_key_iv: "a8c7892ee80ddfd85e05b7168c50dcb9"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.274892 10236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:49.275826 10251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.276049 10236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.276160 10236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "9914f2a9e5c19170a047501f26b989c5"
server_key_iv: "a8c7892ee80ddfd85e05b7168c50dcb9"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.276249 10236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:49.297083 10236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:49.299242 10236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:49.299402 10236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:49.299952 10236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:49.300016 10236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.300081 10236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:49.300112 10236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.307017 10236 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:35001
I20260619 08:14:49.307430 10236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:49.307652 10366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:35001 every 8 connection(s)
I20260619 08:14:49.310863 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.300518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:51719 (local address 127.2.119.254:41783)
0619 08:14:49.300593 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.300597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.300799 (+ 202us) server_negotiation.cc:407] Connection header received
0619 08:14:49.301291 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.301313 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.301327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.301399 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.302219 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.303065 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.304739 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.305049 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.305152 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.308072 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.308073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.308073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.308081 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.309452 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.309871 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.309873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.309873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.309936 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.310203 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.310204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.310204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.310327 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.310399 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.310688 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.310769 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:14:49.311084 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.300501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.194:51719)
0619 08:14:49.300668 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.300708 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.301197 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.301483 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.301486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.301828 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.302123 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.302130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.303175 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.303176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.304635 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.304638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.305154 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.305155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.305259 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.306118 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.306130 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.307968 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.309543 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.309550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.309552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.309775 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.310010 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.310011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.310011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.310127 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.310402 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.310409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.310552 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.310664 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.310826 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260619 08:14:49.311553 10367 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41783
I20260619 08:14:49.311638 10367 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:49.311836 10367 heartbeater.cc:511] Master 127.2.119.254:41783 requested a full tablet report, sending...
I20260619 08:14:49.312235 10024 ts_manager.cc:194] Registered new tserver with Master: 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001)
I20260619 08:14:49.312362 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10236
I20260619 08:14:49.312462 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:49.312676 2527 external_mini_cluster.cc:1428] Setting key b33ed883cfebbb5a8a6d7a350c93a3ef
I20260619 08:14:49.312769 10024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:51719
I20260619 08:14:49.319232 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.312729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:54995 (local address 127.2.119.254:41783)
0619 08:14:49.312769 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.312771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.312885 (+ 114us) server_negotiation.cc:407] Connection header received
0619 08:14:49.312988 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.312988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.312999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.313055 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.313397 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.314111 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.315024 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.315223 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.315272 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.316396 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.316397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.316397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.316405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.317996 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.318418 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.318419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.318419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.318505 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.318723 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.318724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.318724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.318862 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.318913 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.319106 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.319168 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:14:49.319361 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.312687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.194:54995)
0619 08:14:49.312826 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.312836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.312909 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.313059 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.313060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.313252 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.313331 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.313333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.314212 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.314213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.314935 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.314937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.315302 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.315303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.315376 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.315963 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.315969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.316306 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.318091 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.318092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.318092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.318331 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.318578 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.318579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.318580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.318654 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.318930 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.318931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.318987 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.319081 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.319206 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for oryx/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
I20260619 08:14:49.340509 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:41783
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:49.422333 10376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:49.422530 10376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:49.422554 10376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:49.423643 10376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:49.423676 10376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:49.423696 10376 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 08:14:49.423714 10376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:49.425325 10376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.119.254:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:49.425504 10376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:49.425673 10376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:49.431212 10381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:49.431221 10384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.431553 10376 server_base.cc:1034] running on GCE node
W20260619 08:14:49.432049 10382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:49.432330 10376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:49.432643 10376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:49.433831 10376 hybrid_clock.cc:648] HybridClock initialized: now 1781856889433813 us; error 33 us; skew 500 ppm
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:49.437139 10376 init.cc:377] Logged in from keytab as oryx/127.2.119.195@KRBTEST.COM (short username oryx)
I20260619 08:14:49.437810 10376 webserver.cc:460] Webserver started at http://127.2.119.195:36463/ using document root <none> and password file <none>
I20260619 08:14:49.438020 10376 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:49.438093 10376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:49.438237 10376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:49.439476 10376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "348488cf8ca644cc82f1ce6295d9fad7"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "f04d627092145c28c29d19899947db04"
server_key_iv: "26fa4febfa46bff8ffe3b15fbba322ec"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.439920 10376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "348488cf8ca644cc82f1ce6295d9fad7"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "f04d627092145c28c29d19899947db04"
server_key_iv: "26fa4febfa46bff8ffe3b15fbba322ec"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.441416 10376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:14:49.442380 10391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.442591 10376 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:49.442699 10376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "348488cf8ca644cc82f1ce6295d9fad7"
format_stamp: "Formatted at 2026-06-19 08:14:49 on dist-test-slave-d0b5"
server_key: "f04d627092145c28c29d19899947db04"
server_key_iv: "26fa4febfa46bff8ffe3b15fbba322ec"
server_key_version: "encryptionkey@0"
I20260619 08:14:49.442795 10376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:49.451247 10376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:49.453663 10376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:49.453824 10376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:49.454325 10376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:49.454391 10376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.454434 10376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:49.454461 10376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:49.460959 10376 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33307
I20260619 08:14:49.461077 10506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33307 every 8 connection(s)
I20260619 08:14:49.461421 10376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:49.464721 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.454802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:53331 (local address 127.2.119.254:41783)
0619 08:14:49.454855 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.454857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.455132 (+ 275us) server_negotiation.cc:407] Connection header received
0619 08:14:49.455693 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.455694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.455708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.455773 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.456648 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.457517 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.459117 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.459347 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.459433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.462084 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.462086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.462086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.462095 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.463349 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.463747 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.463748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.463748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.463808 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.464066 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.464067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.464067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.464182 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.464255 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.464539 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.464633 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:14:49.464871 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.454792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.195:53331)
0619 08:14:49.454991 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.455034 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.455581 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.455843 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.455847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.456223 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.456504 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.456511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.457618 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.457619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.459022 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.459025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.459441 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.459442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.459583 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.460168 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.460177 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.461984 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.463445 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.463451 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.463453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.463666 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.463885 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.463886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.463886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.463993 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.464256 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.464263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.464397 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.464498 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.464650 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":133,"threads_started":1}
I20260619 08:14:49.465234 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10376
I20260619 08:14:49.465355 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:49.465492 10507 heartbeater.cc:347] Connected to a master server at 127.2.119.254:41783
I20260619 08:14:49.465572 10507 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:49.465574 2527 external_mini_cluster.cc:1428] Setting key da67485ab83e7602e8b733a3b36df12e
I20260619 08:14:49.465749 10507 heartbeater.cc:511] Master 127.2.119.254:41783 requested a full tablet report, sending...
I20260619 08:14:49.466168 10024 ts_manager.cc:194] Registered new tserver with Master: 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:49.466619 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:14:49.466651 10024 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:53331
I20260619 08:14:49.473150 10513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.466643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.2.119.195:37461)
0619 08:14:49.466960 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.466970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.467035 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.467255 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.467256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.467467 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.467529 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.467530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.468248 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.468249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.468903 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.468905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.469414 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.469415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.469487 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.469858 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.469863 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.470264 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.472018 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.472019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.472019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.472252 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.472537 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.472538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.472538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.472623 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.472920 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.472921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.472957 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.473009 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.473086 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":75,"threads_started":1}
I20260619 08:14:49.473232 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.466694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:37461 (local address 127.2.119.254:41783)
0619 08:14:49.466729 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.466731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.467012 (+ 281us) server_negotiation.cc:407] Connection header received
0619 08:14:49.467150 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.467151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.467163 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.467193 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.467624 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.468143 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.468986 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.469291 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.469384 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.470362 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.470364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.470364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.470370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.471931 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.472359 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.472360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.472360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.472419 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.472701 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.472701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.472702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.472852 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.472928 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.473036 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.473116 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:14:49.489409 10092 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:49.483875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.2.119.254:41783)
0619 08:14:49.483937 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.483939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.483948 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:49.483976 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.483977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.483986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.484043 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.484633 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.485115 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.485936 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.486135 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.486187 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.489285 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33540: BlockingRecv error: recv got EOF from 127.0.0.1:33540 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856889, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:49.496634 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.490218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33546 (local address 127.2.119.254:41783)
0619 08:14:49.490271 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.490273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.490306 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:14:49.490363 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.490363 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.490370 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.490419 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.490857 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.491295 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.491916 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.492090 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.492140 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.494339 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.494340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.494340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.494348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.495516 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.495868 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.495870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.495870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.495930 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.496129 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.496129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.496130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.496237 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.496344 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.496439 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.496515 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:14:49.498175 10024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:49.500134 10024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:49.506805 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.2.119.195:33307)
0619 08:14:49.503198 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.503202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.503514 (+ 312us) server_negotiation.cc:407] Connection header received
0619 08:14:49.503554 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.503556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.503569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.503634 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:49.503830 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504560 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.505888 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506364 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506474 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506562 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.506667 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.506706 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":107,"threads_started":1}
I20260619 08:14:49.506794 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33307 (local address 127.0.0.1:54778)
0619 08:14:49.503392 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.503402 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.503452 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.503696 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.503697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.503699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:49.503734 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.503736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504649 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.504649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.505807 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.505809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506476 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.506565 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506581 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.506654 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.506706 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":28,"thread_start_us":68,"threads_started":1}
I20260619 08:14:49.507061 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.2.119.194:35001)
0619 08:14:49.503124 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.503128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.503145 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:14:49.503180 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.503182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.503198 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.503280 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:49.503426 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504537 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.505708 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506462 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506655 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.506923 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.506961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260619 08:14:49.507169 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.2.119.193:39487)
0619 08:14:49.502793 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.502798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.502920 (+ 122us) server_negotiation.cc:407] Connection header received
0619 08:14:49.503016 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.503019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.503037 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.503121 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:49.503277 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504382 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.505853 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506542 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506600 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506668 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.507024 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.507066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260619 08:14:49.507336 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35001 (local address 127.0.0.1:37718)
0619 08:14:49.503021 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.503032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.503085 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.503277 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.503277 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.503280 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:49.503333 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.503334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504622 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.504623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.505596 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.505598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506718 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.506779 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506784 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.506836 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.506883 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":34,"thread_start_us":77,"threads_started":1}
I20260619 08:14:49.506958 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.502480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.0.0.1:58842)
0619 08:14:49.502801 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.502843 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.502925 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.503140 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.503143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.503146 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:49.503214 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.503217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.504474 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.504475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.505766 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.505769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.506700 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.506701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.506793 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.506796 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.506840 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.506876 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":43,"threads_started":1}
I20260619 08:14:49.507771 10441 tablet_service.cc:1467] Processing CreateTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (DEFAULT_TABLE table=test-table [id=debc03bccb264658b61ff2a9614b28a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.507993 10300 tablet_service.cc:1467] Processing CreateTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (DEFAULT_TABLE table=test-table [id=debc03bccb264658b61ff2a9614b28a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.508174 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.508118 10161 tablet_service.cc:1467] Processing CreateTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (DEFAULT_TABLE table=test-table [id=debc03bccb264658b61ff2a9614b28a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.508373 10300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.508445 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.511363 10535 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Bootstrap starting.
I20260619 08:14:49.511965 10536 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Bootstrap starting.
I20260619 08:14:49.512183 10535 tablet_bootstrap.cc:654] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.512338 10537 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Bootstrap starting.
I20260619 08:14:49.512652 10536 tablet_bootstrap.cc:654] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.512735 10535 log.cc:824] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:49.512959 10536 log.cc:824] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:49.513033 10537 tablet_bootstrap.cc:654] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.513398 10537 log.cc:824] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:49.513628 10535 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: No bootstrap required, opened a new log
I20260619 08:14:49.513708 10535 ts_tablet_manager.cc:1397] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:49.513783 10536 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: No bootstrap required, opened a new log
I20260619 08:14:49.513847 10536 ts_tablet_manager.cc:1397] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:14:49.514223 10537 tablet_bootstrap.cc:492] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: No bootstrap required, opened a new log
I20260619 08:14:49.514358 10537 ts_tablet_manager.cc:1397] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:14:49.515602 10535 raft_consensus.cc:348] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.515750 10535 raft_consensus.cc:374] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.515799 10535 raft_consensus.cc:729] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857fcf2ff3a345ddb2a4dfb8f7c4affb, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.515919 10535 consensus_queue.cc:260] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.515971 10536 raft_consensus.cc:348] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.516136 10536 raft_consensus.cc:374] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.516176 10536 raft_consensus.cc:729] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.516206 10535 ts_tablet_manager.cc:1428] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:14:49.516323 10367 heartbeater.cc:503] Master 127.2.119.254:41783 was elected leader, sending a full tablet report...
I20260619 08:14:49.516294 10536 consensus_queue.cc:260] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.516543 10536 ts_tablet_manager.cc:1428] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:14:49.516580 10507 heartbeater.cc:503] Master 127.2.119.254:41783 was elected leader, sending a full tablet report...
I20260619 08:14:49.516988 10537 raft_consensus.cc:348] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.517153 10537 raft_consensus.cc:374] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.517204 10537 raft_consensus.cc:729] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.517334 10537 consensus_queue.cc:260] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.517645 10537 ts_tablet_manager.cc:1428] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:14:49.517710 10227 heartbeater.cc:503] Master 127.2.119.254:41783 was elected leader, sending a full tablet report...
W20260619 08:14:49.558112 10368 tablet.cc:2369] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:49.600240 10542 raft_consensus.cc:482] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:49.600359 10542 raft_consensus.cc:504] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.600656 10542 leader_election.cc:288] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487), 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001)
I20260619 08:14:49.603642 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.600951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:54875 (local address 127.2.119.193:39487)
0619 08:14:49.600996 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.600998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.601009 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:49.601065 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.601066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.601118 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.601186 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:49.601371 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.601901 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.602872 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.603267 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.603321 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.603328 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.603531 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.603573 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:14:49.603665 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.600866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.2.119.195:54875)
0619 08:14:49.600925 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.600938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.600997 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.601205 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.601205 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.601207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:49.601285 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.601286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.601982 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.601983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.602770 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.602772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.603351 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.603352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.603449 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.603453 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.603507 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.603568 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:14:49.603701 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.600952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:49267 (local address 127.2.119.194:35001)
0619 08:14:49.600996 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.600998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.601153 (+ 155us) server_negotiation.cc:407] Connection header received
0619 08:14:49.601204 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.601205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.601215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.601278 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:49.601522 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.602015 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.602943 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.603324 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.603399 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.603428 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.603580 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.603627 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:14:49.603688 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.600868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35001 (local address 127.2.119.195:49267)
0619 08:14:49.601068 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.601079 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.601144 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.601335 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.601336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.601338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:49.601422 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.601424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.602102 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.602103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.602843 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.602845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.603422 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.603423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.603476 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.603479 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.603556 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.603600 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":12,"thread_start_us":122,"threads_started":1}
I20260619 08:14:49.604094 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ca16a4b82d4bf8b4a16bb1c3688a80" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a437dc1556804ee5ad20510dd640f8b3" is_pre_election: true
I20260619 08:14:49.604090 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ca16a4b82d4bf8b4a16bb1c3688a80" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" is_pre_election: true
I20260619 08:14:49.604228 10181 raft_consensus.cc:2454] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 0.
I20260619 08:14:49.604259 10308 raft_consensus.cc:2454] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 0.
I20260619 08:14:49.604506 10394 leader_election.cc:302] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348488cf8ca644cc82f1ce6295d9fad7, 857fcf2ff3a345ddb2a4dfb8f7c4affb; no voters:
I20260619 08:14:49.604691 10542 raft_consensus.cc:2781] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:49.604771 10542 raft_consensus.cc:482] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:49.604802 10542 raft_consensus.cc:3037] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.605849 10542 raft_consensus.cc:504] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.606014 10542 leader_election.cc:288] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 election: Requested vote from peers a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487), 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001)
I20260619 08:14:49.606168 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ca16a4b82d4bf8b4a16bb1c3688a80" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a437dc1556804ee5ad20510dd640f8b3"
I20260619 08:14:49.606160 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ca16a4b82d4bf8b4a16bb1c3688a80" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
I20260619 08:14:49.606252 10181 raft_consensus.cc:3037] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.606281 10308 raft_consensus.cc:3037] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.607031 10181 raft_consensus.cc:2454] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 1.
I20260619 08:14:49.607214 10308 raft_consensus.cc:2454] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 1.
I20260619 08:14:49.607239 10393 leader_election.cc:302] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348488cf8ca644cc82f1ce6295d9fad7, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:49.607357 10542 raft_consensus.cc:2781] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:49.607507 10542 raft_consensus.cc:686] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 LEADER]: Becoming Leader. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Running, Role: LEADER
I20260619 08:14:49.607622 10542 consensus_queue.cc:237] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.608443 10023 catalog_manager.cc:5495] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "348488cf8ca644cc82f1ce6295d9fad7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } health_report { overall_health: HEALTHY } } }
I20260619 08:14:49.627676 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.625351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.2.119.195:33307)
0619 08:14:49.625400 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.625404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.625437 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:14:49.625518 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.625519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.625536 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.625599 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:49.625723 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.626182 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.626899 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.627092 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.627150 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.627294 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:49.627446 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:49.627522 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.627558 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.627576 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:14:49.629521 10181 raft_consensus.cc:1261] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Refusing update from remote peer 348488cf8ca644cc82f1ce6295d9fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:49.629537 10308 raft_consensus.cc:1261] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Refusing update from remote peer 348488cf8ca644cc82f1ce6295d9fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:49.629880 10545 consensus_queue.cc:1035] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:49.630016 10542 consensus_queue.cc:1035] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:49.633109 10549 mvcc.cc:205] Tried to move back new op lower bound from 7298485819919863808 to 7298485819833462784. Current Snapshot: MvccSnapshot[applied={T|T < 7298485819919863808}]
I20260619 08:14:49.633476 10550 mvcc.cc:205] Tried to move back new op lower bound from 7298485819919863808 to 7298485819833462784. Current Snapshot: MvccSnapshot[applied={T|T < 7298485819919863808 or (T in {7298485819919863808})}]
I20260619 08:14:49.634771 10024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:49.634898 10024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:49.635666 10024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Sending DeleteTablet for 3 replicas of tablet 65ca16a4b82d4bf8b4a16bb1c3688a80
I20260619 08:14:49.636030 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:49 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260619 08:14:49.636111 10300 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:49 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:37718
I20260619 08:14:49.636425 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:49 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54778
I20260619 08:14:49.636544 10558 tablet_replica.cc:330] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: stopping tablet replica
I20260619 08:14:49.636634 10560 tablet_replica.cc:330] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: stopping tablet replica
I20260619 08:14:49.636584 10024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 08:14:49.636682 10558 raft_consensus.cc:2229] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:49.636703 10560 raft_consensus.cc:2229] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 LEADER]: Raft consensus shutting down.
W20260619 08:14:49.636731 10024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:49.636778 10558 raft_consensus.cc:2258] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:49.636785 10560 raft_consensus.cc:2258] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:49.637336 10558 ts_tablet_manager.cc:1905] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:49.638127 10441 tablet_service.cc:1467] Processing CreateTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 (DEFAULT_TABLE table=test-table [id=19bd5a2487a24840baab24cefc443300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.638268 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b600de17c74273bb7e1dbdbb0b6d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.638324 10560 ts_tablet_manager.cc:1905] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:49.639709 10536 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Bootstrap starting.
I20260619 08:14:49.639761 10161 tablet_service.cc:1467] Processing CreateTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 (DEFAULT_TABLE table=test-table [id=19bd5a2487a24840baab24cefc443300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.639844 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b600de17c74273bb7e1dbdbb0b6d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.640153 10536 tablet_bootstrap.cc:654] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.640144 10560 ts_tablet_manager.cc:1918] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:49.640544 10560 log.cc:1196] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/65ca16a4b82d4bf8b4a16bb1c3688a80
I20260619 08:14:49.640935 10536 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: No bootstrap required, opened a new log
I20260619 08:14:49.640990 10536 ts_tablet_manager.cc:1397] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.641139 10536 raft_consensus.cc:348] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.641227 10536 raft_consensus.cc:374] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.641249 10536 raft_consensus.cc:729] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.641253 10537 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Bootstrap starting.
I20260619 08:14:49.641319 10536 consensus_queue.cc:260] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.641410 10536 ts_tablet_manager.cc:1428] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:49.641673 10560 ts_tablet_manager.cc:1939] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting consensus metadata
I20260619 08:14:49.641952 10537 tablet_bootstrap.cc:654] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.642416 10009 catalog_manager.cc:4841] TS 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307): tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (table test-table [id=debc03bccb264658b61ff2a9614b28a8]) successfully deleted
I20260619 08:14:49.642554 10537 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: No bootstrap required, opened a new log
I20260619 08:14:49.642616 10537 ts_tablet_manager.cc:1397] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.642751 10537 raft_consensus.cc:348] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.642848 10537 raft_consensus.cc:374] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.642874 10537 raft_consensus.cc:729] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.642913 10537 consensus_queue.cc:260] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.643013 10537 ts_tablet_manager.cc:1428] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:49.637004 10559 tablet_replica.cc:330] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: stopping tablet replica
I20260619 08:14:49.638127 10300 tablet_service.cc:1467] Processing CreateTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 (DEFAULT_TABLE table=test-table [id=19bd5a2487a24840baab24cefc443300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:49.643882 10559 raft_consensus.cc:2229] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:49.643882 10300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b600de17c74273bb7e1dbdbb0b6d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.644023 10559 raft_consensus.cc:2258] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:49.644205 10558 ts_tablet_manager.cc:1918] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:49.644275 10558 log.cc:1196] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/65ca16a4b82d4bf8b4a16bb1c3688a80
I20260619 08:14:49.644533 10558 ts_tablet_manager.cc:1939] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P a437dc1556804ee5ad20510dd640f8b3: Deleting consensus metadata
I20260619 08:14:49.644964 10009 catalog_manager.cc:4841] TS a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487): tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (table test-table [id=debc03bccb264658b61ff2a9614b28a8]) successfully deleted
I20260619 08:14:49.645277 10559 ts_tablet_manager.cc:1905] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:49.645429 10535 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Bootstrap starting.
I20260619 08:14:49.645841 10535 tablet_bootstrap.cc:654] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.646514 10535 tablet_bootstrap.cc:492] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: No bootstrap required, opened a new log
I20260619 08:14:49.646561 10535 ts_tablet_manager.cc:1397] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:49.646710 10535 raft_consensus.cc:348] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.646733 10559 ts_tablet_manager.cc:1918] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:49.646787 10535 raft_consensus.cc:374] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.646809 10535 raft_consensus.cc:729] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857fcf2ff3a345ddb2a4dfb8f7c4affb, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.646816 10559 log.cc:1196] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/65ca16a4b82d4bf8b4a16bb1c3688a80
I20260619 08:14:49.646867 10535 consensus_queue.cc:260] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.646957 10535 ts_tablet_manager.cc:1428] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 08:14:49.647110 10228 tablet.cc:2369] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:49.647130 10559 ts_tablet_manager.cc:1939] T 65ca16a4b82d4bf8b4a16bb1c3688a80 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting consensus metadata
I20260619 08:14:49.647497 10010 catalog_manager.cc:4841] TS 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001): tablet 65ca16a4b82d4bf8b4a16bb1c3688a80 (table test-table [id=debc03bccb264658b61ff2a9614b28a8]) successfully deleted
W20260619 08:14:49.711925 10508 tablet.cc:2369] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:14:49.738483 10555 raft_consensus.cc:482] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:49.738574 10555 raft_consensus.cc:504] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.738783 10555 leader_election.cc:288] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487), 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001)
I20260619 08:14:49.738976 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b600de17c74273bb7e1dbdbb0b6d69" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a437dc1556804ee5ad20510dd640f8b3" is_pre_election: true
I20260619 08:14:49.738970 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b600de17c74273bb7e1dbdbb0b6d69" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" is_pre_election: true
I20260619 08:14:49.739069 10308 raft_consensus.cc:2454] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 0.
I20260619 08:14:49.739073 10181 raft_consensus.cc:2454] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 0.
I20260619 08:14:49.739217 10393 leader_election.cc:302] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348488cf8ca644cc82f1ce6295d9fad7, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:49.739323 10555 raft_consensus.cc:2781] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:49.739377 10555 raft_consensus.cc:482] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:49.739396 10555 raft_consensus.cc:3037] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.740296 10555 raft_consensus.cc:504] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.740432 10555 leader_election.cc:288] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 election: Requested vote from peers a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487), 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001)
I20260619 08:14:49.740639 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b600de17c74273bb7e1dbdbb0b6d69" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a437dc1556804ee5ad20510dd640f8b3"
I20260619 08:14:49.740759 10181 raft_consensus.cc:3037] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.740732 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b600de17c74273bb7e1dbdbb0b6d69" candidate_uuid: "348488cf8ca644cc82f1ce6295d9fad7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
I20260619 08:14:49.740803 10308 raft_consensus.cc:3037] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:49.741508 10181 raft_consensus.cc:2454] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 1.
I20260619 08:14:49.741585 10308 raft_consensus.cc:2454] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 348488cf8ca644cc82f1ce6295d9fad7 in term 1.
I20260619 08:14:49.741695 10393 leader_election.cc:302] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348488cf8ca644cc82f1ce6295d9fad7, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:49.741807 10555 raft_consensus.cc:2781] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:49.741887 10555 raft_consensus.cc:686] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 LEADER]: Becoming Leader. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Running, Role: LEADER
I20260619 08:14:49.741986 10555 consensus_queue.cc:237] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.742643 10024 catalog_manager.cc:5495] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "348488cf8ca644cc82f1ce6295d9fad7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } health_report { overall_health: HEALTHY } } }
Jun 19 08:14:49 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856888, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.254@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:49.784909 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.778053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.2.119.254:41783)
0619 08:14:49.778151 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:49.778153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:49.778206 (+ 53us) server_negotiation.cc:407] Connection header received
0619 08:14:49.778261 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:49.778262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:49.778271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:49.778314 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:49.778783 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.779385 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.780092 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.780315 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.780378 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.782524 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:49.782525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:49.782525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:49.782532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:49.783705 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.784094 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.784095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.784096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.784153 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:49.784345 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:49.784346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:49.784347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:49.784451 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:49.784528 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:49.784708 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:14:49.784826 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":67}
I20260619 08:14:49.784909 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:49.777982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41783 (local address 127.0.0.1:33556)
0619 08:14:49.778124 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:49.778136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:49.778205 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:49.778334 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:49.778335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:49.778658 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:49.778707 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.778709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.779463 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.779464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.780022 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:49.780023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:49.780385 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:49.780385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:49.780426 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:49.780915 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:49.780920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:49.782424 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:49.783800 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.783807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.783810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.783997 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.784219 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:49.784219 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:49.784220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:49.784277 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:49.784518 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:49.784525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:49.784578 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 08:14:49.784664 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:14:49.784778 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260619 08:14:49.786969 10024 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:33556:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 08:14:49.787171 10024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:49.788622 10300 tablet_service.cc:1467] Processing CreateTablet for tablet b21795ca589f43929738d6f2d6ff58c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc4acace136e497cbab80429db36dd55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:49.788795 10300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21795ca589f43929738d6f2d6ff58c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.789443 10441 tablet_service.cc:1467] Processing CreateTablet for tablet b21795ca589f43929738d6f2d6ff58c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc4acace136e497cbab80429db36dd55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:49.789621 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21795ca589f43929738d6f2d6ff58c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.790309 10535 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Bootstrap starting.
I20260619 08:14:49.790774 10535 tablet_bootstrap.cc:654] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.791253 10536 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: Bootstrap starting.
I20260619 08:14:49.791359 10161 tablet_service.cc:1467] Processing CreateTablet for tablet b21795ca589f43929738d6f2d6ff58c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc4acace136e497cbab80429db36dd55]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 08:14:49.791513 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21795ca589f43929738d6f2d6ff58c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:49.791522 10535 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: No bootstrap required, opened a new log
I20260619 08:14:49.791576 10535 ts_tablet_manager.cc:1397] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:49.791601 10536 tablet_bootstrap.cc:654] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.791755 10535 raft_consensus.cc:348] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.791851 10535 raft_consensus.cc:374] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.791870 10535 raft_consensus.cc:729] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857fcf2ff3a345ddb2a4dfb8f7c4affb, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.791919 10535 consensus_queue.cc:260] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.792006 10535 ts_tablet_manager.cc:1428] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:14:49.792129 10541 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:49.792218 10541 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:49.792282 10536 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: No bootstrap required, opened a new log
I20260619 08:14:49.792339 10536 ts_tablet_manager.cc:1397] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.792447 10536 raft_consensus.cc:348] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.792528 10536 raft_consensus.cc:374] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.792562 10536 raft_consensus.cc:729] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.792620 10536 consensus_queue.cc:260] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.792711 10536 ts_tablet_manager.cc:1428] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:49.792794 10555 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:49.792872 10555 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:49.793072 10537 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: Bootstrap starting.
I20260619 08:14:49.793488 10537 tablet_bootstrap.cc:654] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:49.794095 10537 tablet_bootstrap.cc:492] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: No bootstrap required, opened a new log
I20260619 08:14:49.794183 10537 ts_tablet_manager.cc:1397] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:49.794334 10537 raft_consensus.cc:348] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.794409 10537 raft_consensus.cc:374] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:49.794435 10537 raft_consensus.cc:729] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Initialized, Role: FOLLOWER
I20260619 08:14:49.794493 10537 consensus_queue.cc:260] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:49.794584 10537 ts_tablet_manager.cc:1428] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:49.794653 10543 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:49.794763 10543 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:50.134829 10543 raft_consensus.cc:482] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:50.134946 10543 raft_consensus.cc:504] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.135308 10543 leader_election.cc:288] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.138581 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.135509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35001 (local address 127.2.119.193:54725)
0619 08:14:50.135649 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.135661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.135722 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.135975 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.135976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.135980 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:50.136045 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.136047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.136785 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.136786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.137686 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.137688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.138276 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.138277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.138353 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.138357 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.138444 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.138498 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260619 08:14:50.138581 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.135605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54725 (local address 127.2.119.194:35001)
0619 08:14:50.135711 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.135713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.135724 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:14:50.135821 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.135822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.135834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.135912 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:50.136129 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.136677 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.137790 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.138181 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.138270 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.138279 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.138473 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.138516 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":54,"threads_started":1}
I20260619 08:14:50.138823 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21795ca589f43929738d6f2d6ff58c3" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" is_pre_election: true
I20260619 08:14:50.138943 10308 raft_consensus.cc:2454] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.138988 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.135680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:36689 (local address 127.2.119.195:33307)
0619 08:14:50.135795 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.135797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.135804 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:50.135827 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.135828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.135838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.135916 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:14:50.136081 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.136678 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.137932 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.138355 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.138476 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.138484 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.138861 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.138913 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":46,"threads_started":1}
I20260619 08:14:50.139026 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.135524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33307 (local address 127.2.119.193:36689)
0619 08:14:50.135668 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.135680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.135744 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.135922 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.135923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.135925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:14:50.135995 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.135997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.136770 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.136771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.137836 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.137838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.138517 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.138518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.138589 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.138593 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.138832 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.138888 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":59,"thread_start_us":66,"threads_started":1}
I20260619 08:14:50.139297 10114 leader_election.cc:302] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857fcf2ff3a345ddb2a4dfb8f7c4affb, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.139394 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21795ca589f43929738d6f2d6ff58c3" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7" is_pre_election: true
I20260619 08:14:50.139509 10543 raft_consensus.cc:2781] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:50.139544 10543 raft_consensus.cc:482] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:50.139531 10461 raft_consensus.cc:2454] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.139575 10543 raft_consensus.cc:3037] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.140323 10543 raft_consensus.cc:504] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.140487 10543 leader_election.cc:288] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Requested vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.140686 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21795ca589f43929738d6f2d6ff58c3" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
I20260619 08:14:50.140776 10308 raft_consensus.cc:3037] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.140789 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21795ca589f43929738d6f2d6ff58c3" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7"
I20260619 08:14:50.140882 10461 raft_consensus.cc:3037] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.141450 10308 raft_consensus.cc:2454] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.141604 10114 leader_election.cc:302] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857fcf2ff3a345ddb2a4dfb8f7c4affb, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.141700 10543 raft_consensus.cc:2781] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:50.141724 10461 raft_consensus.cc:2454] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.141827 10543 raft_consensus.cc:686] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [term 1 LEADER]: Becoming Leader. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Running, Role: LEADER
I20260619 08:14:50.141896 10543 consensus_queue.cc:237] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.142251 10578 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: TxnStatusTablet state changed. Reason: New leader a437dc1556804ee5ad20510dd640f8b3. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.142364 10578 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:50.142532 10580 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 08:14:50.142488 10024 catalog_manager.cc:5495] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:50.142634 10580 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 08:14:50.235020 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.232063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.0.0.1:58850)
0619 08:14:50.232169 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.232188 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.232237 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.232510 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.232511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.232512 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:50.232581 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.232583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.233409 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.233410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.234020 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.234022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.234406 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.234406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.234478 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.234493 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.234821 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.234832 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.234893 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.234946 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 08:14:50.235028 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.232166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.2.119.193:39487)
0619 08:14:50.232318 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.232321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.232329 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:50.232379 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.232379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.232405 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.232510 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:50.232657 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.233238 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.234112 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.234328 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.234405 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.234636 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.234741 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.234837 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.234897 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.234921 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260619 08:14:50.236548 10308 raft_consensus.cc:1261] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.236738 10461 raft_consensus.cc:1261] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.236924 10543 consensus_queue.cc:1035] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.237246 10543 consensus_queue.cc:1035] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.237627 10541 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: TxnStatusTablet state changed. Reason: New leader a437dc1556804ee5ad20510dd640f8b3. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.237720 10541 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:50.237948 10555 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: TxnStatusTablet state changed. Reason: New leader a437dc1556804ee5ad20510dd640f8b3. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.238035 10555 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:50.238739 10578 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.238826 10578 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:50.238770 10543 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.239167 10543 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P a437dc1556804ee5ad20510dd640f8b3: This TxnStatusTablet replica's current role is: LEADER
I20260619 08:14:50.239348 10541 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.239434 10541 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:50.239542 10587 mvcc.cc:205] Tried to move back new op lower bound from 7298485822406590464 to 7298485822021840896. Current Snapshot: MvccSnapshot[applied={T|T < 7298485822406590464}]
I20260619 08:14:50.239794 10555 tablet_replica.cc:439] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } } }
I20260619 08:14:50.239871 10555 tablet_replica.cc:442] T b21795ca589f43929738d6f2d6ff58c3 P 348488cf8ca644cc82f1ce6295d9fad7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 08:14:50.248121 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.242408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:47949 (local address 127.2.119.193:39487)
0619 08:14:50.242447 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.242449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.245509 (+ 3060us) server_negotiation.cc:407] Connection header received
0619 08:14:50.245590 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.245591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.245602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.245657 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:50.245826 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.246372 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.247458 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.247659 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.247707 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.247898 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.247925 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.247963 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.248044 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.248062 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:14:50.248183 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.242328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.2.119.195:47949)
0619 08:14:50.245414 (+ 3086us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.245428 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.245509 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.245699 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.245700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.245702 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:50.245758 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.245760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.246467 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.246468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.247362 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.247365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.247731 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.247732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.247797 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.247814 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.247984 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.247986 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.248043 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.248097 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3047,"thread_start_us":45,"threads_started":1}
I20260619 08:14:50.251034 10181 raft_consensus.cc:1261] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Refusing update from remote peer 348488cf8ca644cc82f1ce6295d9fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.251070 10308 raft_consensus.cc:1261] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Refusing update from remote peer 348488cf8ca644cc82f1ce6295d9fad7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.251305 10596 consensus_queue.cc:1035] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.251425 10555 consensus_queue.cc:1035] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.260690 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.257718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:40015 (local address 127.2.119.195:33307)
0619 08:14:50.257754 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.257756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.257814 (+ 58us) server_negotiation.cc:407] Connection header received
0619 08:14:50.257879 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.257880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.257889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.257941 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:50.258039 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.258792 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.259930 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.260233 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.260301 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.260445 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.260480 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.260551 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.260583 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.260608 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:14:50.260913 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.257582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33307 (local address 127.2.119.193:40015)
0619 08:14:50.257751 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.257762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.257822 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.257942 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.257942 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.257944 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:50.257997 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.257999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.258866 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.258867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.259860 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.259863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.260308 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.260308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.260357 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.260371 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.260537 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.260539 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.260566 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.260608 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260619 08:14:50.269261 10023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.269443 10023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.270170 10023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Sending DeleteTablet for 3 replicas of tablet c0b600de17c74273bb7e1dbdbb0b6d69
I20260619 08:14:50.270429 10300 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:37718
I20260619 08:14:50.270466 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260619 08:14:50.270684 10603 tablet_replica.cc:330] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: stopping tablet replica
I20260619 08:14:50.270744 10603 raft_consensus.cc:2229] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.270742 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b600de17c74273bb7e1dbdbb0b6d69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54778
I20260619 08:14:50.270779 10603 raft_consensus.cc:2258] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.270910 10602 tablet_replica.cc:330] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: stopping tablet replica
I20260619 08:14:50.270959 10602 raft_consensus.cc:2229] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.270991 10602 raft_consensus.cc:2258] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.271040 10603 ts_tablet_manager.cc:1905] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.271203 10602 ts_tablet_manager.cc:1905] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.272967 10603 ts_tablet_manager.cc:1918] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:50.272979 10602 ts_tablet_manager.cc:1918] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:50.273030 10602 log.cc:1196] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/c0b600de17c74273bb7e1dbdbb0b6d69
I20260619 08:14:50.273033 10603 log.cc:1196] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/c0b600de17c74273bb7e1dbdbb0b6d69
I20260619 08:14:50.273247 10602 ts_tablet_manager.cc:1939] T c0b600de17c74273bb7e1dbdbb0b6d69 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting consensus metadata
I20260619 08:14:50.273267 10603 ts_tablet_manager.cc:1939] T c0b600de17c74273bb7e1dbdbb0b6d69 P a437dc1556804ee5ad20510dd640f8b3: Deleting consensus metadata
I20260619 08:14:50.273895 10010 catalog_manager.cc:4841] TS 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001): tablet c0b600de17c74273bb7e1dbdbb0b6d69 (table test-table [id=19bd5a2487a24840baab24cefc443300]) successfully deleted
I20260619 08:14:50.274116 10009 catalog_manager.cc:4841] TS a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487): tablet c0b600de17c74273bb7e1dbdbb0b6d69 (table test-table [id=19bd5a2487a24840baab24cefc443300]) successfully deleted
I20260619 08:14:50.275115 10604 tablet_replica.cc:330] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: stopping tablet replica
I20260619 08:14:50.275198 10604 raft_consensus.cc:2229] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:50.275269 10604 raft_consensus.cc:2258] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.275518 10604 ts_tablet_manager.cc:1905] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.277020 10604 ts_tablet_manager.cc:1918] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:50.277086 10604 log.cc:1196] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/c0b600de17c74273bb7e1dbdbb0b6d69
I20260619 08:14:50.277349 10604 ts_tablet_manager.cc:1939] T c0b600de17c74273bb7e1dbdbb0b6d69 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting consensus metadata
I20260619 08:14:50.277668 10009 catalog_manager.cc:4841] TS 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307): tablet c0b600de17c74273bb7e1dbdbb0b6d69 (table test-table [id=19bd5a2487a24840baab24cefc443300]) successfully deleted
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[9979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:14:50.295979 10612 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:50.289566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.2.119.254:41783)
0619 08:14:50.289730 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.289732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.289766 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:14:50.289835 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.289836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.289847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.289910 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:50.290638 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.291207 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.292037 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.292394 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.292459 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.295862 (+ 3403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33558: BlockingRecv error: recv got EOF from 127.0.0.1:33558 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[9979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM
I20260619 08:14:50.304992 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.296904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.2.119.254:41783)
0619 08:14:50.296963 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.296965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.297131 (+ 166us) server_negotiation.cc:407] Connection header received
0619 08:14:50.297226 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.297227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.297243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.297394 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:50.297872 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.298441 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.299225 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.299442 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.299538 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.302042 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:50.302043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:50.302043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:50.302050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:50.303621 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:50.304210 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:50.304212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:50.304212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:50.304272 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:50.304521 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:50.304522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:50.304522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:50.304648 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:50.304734 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.304847 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.304916 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:14:50.307037 10023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:50.307224 10023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:50.308630 10161 tablet_service.cc:1467] Processing CreateTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb (DEFAULT_TABLE table=test-table [id=35b7ef605003493db840cf18fa6d746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.308676 10441 tablet_service.cc:1467] Processing CreateTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb (DEFAULT_TABLE table=test-table [id=35b7ef605003493db840cf18fa6d746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.308772 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4ad61ff84aa4f22acffbbcc48e391eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.308807 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4ad61ff84aa4f22acffbbcc48e391eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.310292 10618 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Bootstrap starting.
I20260619 08:14:50.310686 10618 tablet_bootstrap.cc:654] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.311012 10619 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Bootstrap starting.
I20260619 08:14:50.311079 10300 tablet_service.cc:1467] Processing CreateTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb (DEFAULT_TABLE table=test-table [id=35b7ef605003493db840cf18fa6d746e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.311197 10300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4ad61ff84aa4f22acffbbcc48e391eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.311352 10618 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: No bootstrap required, opened a new log
I20260619 08:14:50.311424 10618 ts_tablet_manager.cc:1397] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:50.311419 10619 tablet_bootstrap.cc:654] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.311553 10618 raft_consensus.cc:348] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.311638 10618 raft_consensus.cc:374] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.311666 10618 raft_consensus.cc:729] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.311718 10618 consensus_queue.cc:260] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.311830 10618 ts_tablet_manager.cc:1428] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:50.312070 10619 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: No bootstrap required, opened a new log
I20260619 08:14:50.312119 10619 ts_tablet_manager.cc:1397] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:50.312258 10619 raft_consensus.cc:348] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.312341 10619 raft_consensus.cc:374] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.312381 10619 raft_consensus.cc:729] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.312424 10619 consensus_queue.cc:260] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.312537 10619 ts_tablet_manager.cc:1428] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:50.312676 10622 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Bootstrap starting.
I20260619 08:14:50.313050 10622 tablet_bootstrap.cc:654] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.313601 10622 tablet_bootstrap.cc:492] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: No bootstrap required, opened a new log
I20260619 08:14:50.313647 10622 ts_tablet_manager.cc:1397] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:50.313890 10622 raft_consensus.cc:348] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.314035 10622 raft_consensus.cc:374] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.314075 10622 raft_consensus.cc:729] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857fcf2ff3a345ddb2a4dfb8f7c4affb, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.314131 10622 consensus_queue.cc:260] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.314236 10622 ts_tablet_manager.cc:1428] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:50.409013 10590 raft_consensus.cc:482] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:50.409125 10590 raft_consensus.cc:504] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.409391 10590 leader_election.cc:288] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.409662 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4ad61ff84aa4f22acffbbcc48e391eb" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" is_pre_election: true
I20260619 08:14:50.409660 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4ad61ff84aa4f22acffbbcc48e391eb" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7" is_pre_election: true
I20260619 08:14:50.409761 10308 raft_consensus.cc:2454] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.409761 10461 raft_consensus.cc:2454] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.409950 10113 leader_election.cc:302] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 348488cf8ca644cc82f1ce6295d9fad7, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.410094 10590 raft_consensus.cc:2781] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:50.410138 10590 raft_consensus.cc:482] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:50.410163 10590 raft_consensus.cc:3037] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.410866 10590 raft_consensus.cc:504] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.411015 10590 leader_election.cc:288] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Requested vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.411192 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4ad61ff84aa4f22acffbbcc48e391eb" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
I20260619 08:14:50.411228 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4ad61ff84aa4f22acffbbcc48e391eb" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7"
I20260619 08:14:50.411278 10308 raft_consensus.cc:3037] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.411289 10461 raft_consensus.cc:3037] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.412066 10461 raft_consensus.cc:2454] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.412067 10308 raft_consensus.cc:2454] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.412258 10114 leader_election.cc:302] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857fcf2ff3a345ddb2a4dfb8f7c4affb, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.412374 10590 raft_consensus.cc:2781] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:50.412433 10590 raft_consensus.cc:686] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 1 LEADER]: Becoming Leader. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Running, Role: LEADER
I20260619 08:14:50.412482 10590 consensus_queue.cc:237] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.413089 10024 catalog_manager.cc:5495] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:50.429741 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.427317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.2.119.193:39487)
0619 08:14:50.427393 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.427395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.427405 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:14:50.427438 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.427439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.427450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.427516 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:50.427656 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.428137 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.428928 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.429164 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.429246 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.429461 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.429499 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.429559 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.429624 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.429648 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:14:50.430776 10461 raft_consensus.cc:1261] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.430784 10308 raft_consensus.cc:1261] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.431026 10590 consensus_queue.cc:1035] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.431129 10543 consensus_queue.cc:1035] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.434355 10024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.434454 10024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.435135 10024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Sending DeleteTablet for 3 replicas of tablet f4ad61ff84aa4f22acffbbcc48e391eb
I20260619 08:14:50.435374 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260619 08:14:50.435374 10300 tablet_service.cc:1514] Processing DeleteTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:37718
I20260619 08:14:50.435460 10603 tablet_replica.cc:330] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: stopping tablet replica
I20260619 08:14:50.435504 10603 raft_consensus.cc:2229] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:50.435535 10602 tablet_replica.cc:330] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: stopping tablet replica
I20260619 08:14:50.435593 10603 raft_consensus.cc:2258] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.435600 10602 raft_consensus.cc:2229] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.435645 10602 raft_consensus.cc:2258] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.435657 10024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:14:50.435806 10024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:14:50.435901 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet f4ad61ff84aa4f22acffbbcc48e391eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54778
I20260619 08:14:50.436049 10604 tablet_replica.cc:330] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: stopping tablet replica
I20260619 08:14:50.436134 10604 raft_consensus.cc:2229] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.436182 10604 raft_consensus.cc:2258] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.436175 10602 ts_tablet_manager.cc:1905] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.436515 10603 ts_tablet_manager.cc:1905] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.437238 10300 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc236aad25a49fc982db06fe199ff50 (DEFAULT_TABLE table=test-table [id=25edbf15fa134032a375a9ba70098ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.437369 10300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc236aad25a49fc982db06fe199ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.438066 10602 ts_tablet_manager.cc:1918] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:50.438128 10602 log.cc:1196] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/f4ad61ff84aa4f22acffbbcc48e391eb
I20260619 08:14:50.438210 10603 ts_tablet_manager.cc:1918] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:50.438261 10603 log.cc:1196] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/f4ad61ff84aa4f22acffbbcc48e391eb
I20260619 08:14:50.438350 10602 ts_tablet_manager.cc:1939] T f4ad61ff84aa4f22acffbbcc48e391eb P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting consensus metadata
I20260619 08:14:50.438472 10603 ts_tablet_manager.cc:1939] T f4ad61ff84aa4f22acffbbcc48e391eb P a437dc1556804ee5ad20510dd640f8b3: Deleting consensus metadata
I20260619 08:14:50.438653 10622 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Bootstrap starting.
I20260619 08:14:50.438678 10010 catalog_manager.cc:4841] TS 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001): tablet f4ad61ff84aa4f22acffbbcc48e391eb (table test-table [id=35b7ef605003493db840cf18fa6d746e]) successfully deleted
I20260619 08:14:50.439062 10622 tablet_bootstrap.cc:654] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.439560 10604 ts_tablet_manager.cc:1905] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.441028 10604 ts_tablet_manager.cc:1918] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:14:50.441093 10604 log.cc:1196] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/f4ad61ff84aa4f22acffbbcc48e391eb
I20260619 08:14:50.441315 10604 ts_tablet_manager.cc:1939] T f4ad61ff84aa4f22acffbbcc48e391eb P 348488cf8ca644cc82f1ce6295d9fad7: Deleting consensus metadata
I20260619 08:14:50.441483 10009 catalog_manager.cc:4841] TS a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487): tablet f4ad61ff84aa4f22acffbbcc48e391eb (table test-table [id=35b7ef605003493db840cf18fa6d746e]) successfully deleted
I20260619 08:14:50.441541 10161 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc236aad25a49fc982db06fe199ff50 (DEFAULT_TABLE table=test-table [id=25edbf15fa134032a375a9ba70098ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.441643 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc236aad25a49fc982db06fe199ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.441682 10441 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc236aad25a49fc982db06fe199ff50 (DEFAULT_TABLE table=test-table [id=25edbf15fa134032a375a9ba70098ba6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:14:50.441771 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc236aad25a49fc982db06fe199ff50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:50.441803 10622 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: No bootstrap required, opened a new log
I20260619 08:14:50.441843 10622 ts_tablet_manager.cc:1397] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 08:14:50.441993 10622 raft_consensus.cc:348] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.442065 10622 raft_consensus.cc:374] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.442085 10622 raft_consensus.cc:729] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857fcf2ff3a345ddb2a4dfb8f7c4affb, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.442127 10622 consensus_queue.cc:260] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.442216 10622 ts_tablet_manager.cc:1428] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:14:50.442847 10619 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Bootstrap starting.
I20260619 08:14:50.442843 10618 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: Bootstrap starting.
I20260619 08:14:50.443120 10618 tablet_bootstrap.cc:654] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.443132 10619 tablet_bootstrap.cc:654] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:50.443522 10009 catalog_manager.cc:4841] TS 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307): tablet f4ad61ff84aa4f22acffbbcc48e391eb (table test-table [id=35b7ef605003493db840cf18fa6d746e]) successfully deleted
I20260619 08:14:50.443678 10618 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: No bootstrap required, opened a new log
I20260619 08:14:50.443724 10618 ts_tablet_manager.cc:1397] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:50.443852 10618 raft_consensus.cc:348] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.443921 10618 raft_consensus.cc:374] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.443930 10619 tablet_bootstrap.cc:492] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: No bootstrap required, opened a new log
I20260619 08:14:50.443940 10618 raft_consensus.cc:729] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.443971 10619 ts_tablet_manager.cc:1397] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:50.444007 10618 consensus_queue.cc:260] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.444067 10619 raft_consensus.cc:348] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.444098 10618 ts_tablet_manager.cc:1428] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:50.444124 10619 raft_consensus.cc:374] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:50.444149 10619 raft_consensus.cc:729] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348488cf8ca644cc82f1ce6295d9fad7, State: Initialized, Role: FOLLOWER
I20260619 08:14:50.444186 10619 consensus_queue.cc:260] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.444269 10619 ts_tablet_manager.cc:1428] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:50.519857 10590 raft_consensus.cc:482] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:14:50.519968 10590 raft_consensus.cc:504] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.520115 10590 leader_election.cc:288] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.520362 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc236aad25a49fc982db06fe199ff50" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" is_pre_election: true
I20260619 08:14:50.520368 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc236aad25a49fc982db06fe199ff50" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7" is_pre_election: true
I20260619 08:14:50.520459 10308 raft_consensus.cc:2454] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.520501 10461 raft_consensus.cc:2454] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 0.
I20260619 08:14:50.520653 10114 leader_election.cc:302] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857fcf2ff3a345ddb2a4dfb8f7c4affb, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.520778 10590 raft_consensus.cc:2781] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:14:50.520828 10590 raft_consensus.cc:482] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:14:50.520848 10590 raft_consensus.cc:3037] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.521754 10590 raft_consensus.cc:504] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.521891 10590 leader_election.cc:288] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Requested vote from peers 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001), 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307)
I20260619 08:14:50.522145 10461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc236aad25a49fc982db06fe199ff50" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "348488cf8ca644cc82f1ce6295d9fad7"
I20260619 08:14:50.522173 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc236aad25a49fc982db06fe199ff50" candidate_uuid: "a437dc1556804ee5ad20510dd640f8b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb"
I20260619 08:14:50.522239 10308 raft_consensus.cc:3037] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.522238 10461 raft_consensus.cc:3037] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:50.523027 10461 raft_consensus.cc:2454] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.523025 10308 raft_consensus.cc:2454] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a437dc1556804ee5ad20510dd640f8b3 in term 1.
I20260619 08:14:50.523213 10114 leader_election.cc:302] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 857fcf2ff3a345ddb2a4dfb8f7c4affb, a437dc1556804ee5ad20510dd640f8b3; no voters:
I20260619 08:14:50.523343 10590 raft_consensus.cc:2781] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:50.523422 10590 raft_consensus.cc:686] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 1 LEADER]: Becoming Leader. State: Replica: a437dc1556804ee5ad20510dd640f8b3, State: Running, Role: LEADER
I20260619 08:14:50.523476 10590 consensus_queue.cc:237] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } }
I20260619 08:14:50.524161 10024 catalog_manager.cc:5495] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "a437dc1556804ee5ad20510dd640f8b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a437dc1556804ee5ad20510dd640f8b3" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 39487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 } health_report { overall_health: UNKNOWN } } }
I20260619 08:14:50.545480 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.542953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:47555 (local address 127.2.119.193:39487)
0619 08:14:50.542984 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:50.542986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:50.543042 (+ 56us) server_negotiation.cc:407] Connection header received
0619 08:14:50.543103 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:50.543103 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:50.543111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:50.543149 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:14:50.543381 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.543870 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.544766 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.544939 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.544991 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.545145 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.545206 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.545282 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:50.545370 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 08:14:50.545397 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11}
I20260619 08:14:50.545549 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.542926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.2.119.193:47555)
0619 08:14:50.542982 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.542995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:50.543050 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:50.543158 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:50.543159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:50.543160 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 08:14:50.543226 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.543227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.543955 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.543956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.544685 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:50.544687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:50.545013 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:50.545013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:50.545070 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:50.545077 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 08:14:50.545286 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 08:14:50.545288 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:14:50.545359 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:14:50.545403 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:14:50.547588 10461 raft_consensus.cc:1261] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.547624 10308 raft_consensus.cc:1261] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Refusing update from remote peer a437dc1556804ee5ad20510dd640f8b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:14:50.547852 10578 consensus_queue.cc:1035] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "348488cf8ca644cc82f1ce6295d9fad7" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.547966 10578 consensus_queue.cc:1035] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857fcf2ff3a345ddb2a4dfb8f7c4affb" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:14:50.557504 10024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.557607 10024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:14:50.558226 10024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 105de698bbb44bfdb8934961e77ffd76: Sending DeleteTablet for 3 replicas of tablet 5dc236aad25a49fc982db06fe199ff50
I20260619 08:14:50.558439 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc236aad25a49fc982db06fe199ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58842
I20260619 08:14:50.558454 10300 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc236aad25a49fc982db06fe199ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:37718
I20260619 08:14:50.558555 10603 tablet_replica.cc:330] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: stopping tablet replica
I20260619 08:14:50.558570 10602 tablet_replica.cc:330] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: stopping tablet replica
I20260619 08:14:50.558524 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc236aad25a49fc982db06fe199ff50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:14:50 UTC) from {username='oryx', principal='oryx/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54778
I20260619 08:14:50.558619 10603 raft_consensus.cc:2229] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:14:50.558620 10602 raft_consensus.cc:2229] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.558657 10602 raft_consensus.cc:2258] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.558671 10603 raft_consensus.cc:2258] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.558745 10604 tablet_replica.cc:330] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: stopping tablet replica
I20260619 08:14:50.558804 10604 raft_consensus.cc:2229] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:14:50.558840 10604 raft_consensus.cc:2258] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:14:50.559036 10602 ts_tablet_manager.cc:1905] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.559240 10603 ts_tablet_manager.cc:1905] T 5dc236aad25a49fc982db06fe199ff50 P a437dc1556804ee5ad20510dd640f8b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:14:50.559263 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10096
I20260619 08:14:50.560777 10602 ts_tablet_manager.cc:1918] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:50.560837 10602 log.cc:1196] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/5dc236aad25a49fc982db06fe199ff50
I20260619 08:14:50.561098 10602 ts_tablet_manager.cc:1939] T 5dc236aad25a49fc982db06fe199ff50 P 857fcf2ff3a345ddb2a4dfb8f7c4affb: Deleting consensus metadata
I20260619 08:14:50.561504 10010 catalog_manager.cc:4841] TS 857fcf2ff3a345ddb2a4dfb8f7c4affb (127.2.119.194:35001): tablet 5dc236aad25a49fc982db06fe199ff50 (table test-table [id=25edbf15fa134032a375a9ba70098ba6]) successfully deleted
I20260619 08:14:50.565506 10604 ts_tablet_manager.cc:1905] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:14:50.566419 10009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:39487 (error 108)
I20260619 08:14:50.566576 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10236
I20260619 08:14:50.567313 10604 ts_tablet_manager.cc:1918] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 08:14:50.567377 10604 log.cc:1196] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/5dc236aad25a49fc982db06fe199ff50
I20260619 08:14:50.567593 10604 ts_tablet_manager.cc:1939] T 5dc236aad25a49fc982db06fe199ff50 P 348488cf8ca644cc82f1ce6295d9fad7: Deleting consensus metadata
I20260619 08:14:50.567756 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:50.567542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:39487 (local address 127.0.0.1:58862)
0619 08:14:50.567672 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:50.567724 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:39487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":57,"threads_started":1}
W20260619 08:14:50.567927 10009 catalog_manager.cc:4584] TS a437dc1556804ee5ad20510dd640f8b3 (127.2.119.193:39487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5dc236aad25a49fc982db06fe199ff50: Network error: Client connection negotiation failed: client connection to 127.2.119.193:39487: connect: Connection refused (error 111)
I20260619 08:14:50.568252 10009 catalog_manager.cc:4841] TS 348488cf8ca644cc82f1ce6295d9fad7 (127.2.119.195:33307): tablet 5dc236aad25a49fc982db06fe199ff50 (table test-table [id=25edbf15fa134032a375a9ba70098ba6]) successfully deleted
I20260619 08:14:50.572542 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10376
I20260619 08:14:50.577877 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 9992
2026-06-19T08:14:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2945 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[10640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[10640](info): set up 2 sockets
Jun 19 08:14:50 dist-test-slave-d0b5 krb5kdc[10640](info): commencing operation
krb5kdc: starting...
W20260619 08:14:51.627077 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:51 dist-test-slave-d0b5 krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:51.752575 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41665
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:51.834802 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:51.835075 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:51.835124 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:51.836318 10654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:51.836387 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:51.836413 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:51.836444 10654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:51.836468 10654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:51.838079 10654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:51.838366 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:51.838596 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:51.843494 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:51.843699 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:51.843780 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:51.843856 10654 server_base.cc:1034] running on GCE node
I20260619 08:14:51.844080 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:51.844338 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:51.845520 10654 hybrid_clock.cc:648] HybridClock initialized: now 1781856891845502 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T08:14:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1271 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:14:51 dist-test-slave-d0b5 krb5kdc[10667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:14:51 dist-test-slave-d0b5 krb5kdc[10667](info): set up 2 sockets
Jun 19 08:14:51 dist-test-slave-d0b5 krb5kdc[10667](info): commencing operation
krb5kdc: starting...
W20260619 08:14:52.894977 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:14:52 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:14:53.016297 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39287
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:39287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:53.097514 10680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.097772 10680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.097808 10680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.098981 10680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:53.099051 10680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.099076 10680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.099097 10680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:53.099123 10680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:53.100625 10680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:39287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.100893 10680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.101125 10680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.106176 10688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.106230 10680 server_base.cc:1034] running on GCE node
W20260619 08:14:53.106207 10685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.106176 10686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.106706 10680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.106952 10680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.108091 10680 hybrid_clock.cc:648] HybridClock initialized: now 1781856893108056 us; error 51 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.114943 10680 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:53.115433 10680 webserver.cc:460] Webserver started at http://127.2.119.254:46531/ using document root <none> and password file <none>
I20260619 08:14:53.115679 10680 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.115749 10680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.115892 10680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:53.116928 10680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "fb8fa01608564539bb14b012fc1798f9"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "142d1056c911ca1c0943727778a3cb14"
server_key_iv: "aa38011956473133091d6b940ee0078d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.117341 10680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "fb8fa01608564539bb14b012fc1798f9"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "142d1056c911ca1c0943727778a3cb14"
server_key_iv: "aa38011956473133091d6b940ee0078d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.118680 10680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:53.119428 10695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.119649 10680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:53.119750 10680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "fb8fa01608564539bb14b012fc1798f9"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "142d1056c911ca1c0943727778a3cb14"
server_key_iv: "aa38011956473133091d6b940ee0078d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.119833 10680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.130704 10680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.133090 10680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.133265 10680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.137284 10680 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:39287
I20260619 08:14:53.137370 10748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:39287 every 8 connection(s)
I20260619 08:14:53.137813 10680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:53.138653 10749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:14:53.140899 10749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Bootstrap starting.
I20260619 08:14:53.141223 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10680
I20260619 08:14:53.141368 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:14:53.141578 2527 external_mini_cluster.cc:1428] Setting key 3e073a7ce33be0362369585d5289e13e
I20260619 08:14:53.141647 10749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Neither blocks nor log segments found. Creating new log.
I20260619 08:14:53.141955 10749 log.cc:824] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:53.142813 10749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: No bootstrap required, opened a new log
I20260619 08:14:53.145079 10749 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.145241 10749 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:14:53.145268 10749 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8fa01608564539bb14b012fc1798f9, State: Initialized, Role: FOLLOWER
I20260619 08:14:53.145414 10749 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.145509 10749 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:53.145553 10749 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:53.145620 10749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:14:53.146159 10749 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.146283 10749 leader_election.cc:302] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8fa01608564539bb14b012fc1798f9; no voters:
I20260619 08:14:53.146440 10749 leader_election.cc:288] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:14:53.146466 10754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:14:53.146574 10754 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 LEADER]: Becoming Leader. State: Replica: fb8fa01608564539bb14b012fc1798f9, State: Running, Role: LEADER
I20260619 08:14:53.146654 10754 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.146693 10749 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:53.146976 10754 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8fa01608564539bb14b012fc1798f9. Latest consensus state: current_term: 1 leader_uuid: "fb8fa01608564539bb14b012fc1798f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } } }
I20260619 08:14:53.147058 10754 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:53.146987 10755 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8fa01608564539bb14b012fc1798f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } } }
I20260619 08:14:53.147250 10755 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:53.147426 10760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:53.147990 10760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:53.149911 10760 catalog_manager.cc:1347] Generated new cluster ID: a6a9fd7b04bd4fe89d9464c0cb93bec9
I20260619 08:14:53.149966 10760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:53.153450 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.142212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.2.119.254:39287)
0619 08:14:53.142400 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:53.142407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:53.142431 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:14:53.143016 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:53.143025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:53.143083 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:53.143244 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:53.143857 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.145361 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.146298 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.146983 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.147093 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.150171 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:53.150174 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:53.150175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:53.150186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:53.151896 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.152284 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.152287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.152287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.152376 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.152601 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.152602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.152603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.152829 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:53.153020 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:53.153093 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 08:14:53.153227 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260619 08:14:53.161912 10760 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:14:53.162549 10760 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:53.171239 10760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Generated new TSK 0
I20260619 08:14:53.171428 10760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:14:53.201036 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:53.283650 10775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.283895 10775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.283932 10775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.285815 10775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.285882 10775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.285926 10775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:53.287381 10775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.287658 10775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.287895 10775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.292966 10783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.293092 10780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.293136 10775 server_base.cc:1034] running on GCE node
W20260619 08:14:53.292966 10781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.293427 10775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.293637 10775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.294814 10775 hybrid_clock.cc:648] HybridClock initialized: now 1781856893294801 us; error 39 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.297254 10775 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:14:53.297700 10775 webserver.cc:460] Webserver started at http://127.2.119.193:42425/ using document root <none> and password file <none>
I20260619 08:14:53.297924 10775 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.297992 10775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.298139 10775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:53.299294 10775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "fdc25072a20c4147a5b5624598924e2f"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "32c0fbb960b932bb6a528d39f87524bd"
server_key_iv: "c34157e1b075afc894cca848d7e7003d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.299660 10775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "fdc25072a20c4147a5b5624598924e2f"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "32c0fbb960b932bb6a528d39f87524bd"
server_key_iv: "c34157e1b075afc894cca848d7e7003d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.300976 10775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 08:14:53.301811 10790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.302011 10775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:14:53.302114 10775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "fdc25072a20c4147a5b5624598924e2f"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "32c0fbb960b932bb6a528d39f87524bd"
server_key_iv: "c34157e1b075afc894cca848d7e7003d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.302188 10775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.318593 10775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.320923 10775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.321122 10775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.321470 10775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:53.321858 10775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:53.321918 10775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.321959 10775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:53.321990 10775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.328465 10775 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34611
I20260619 08:14:53.328510 10905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34611 every 8 connection(s)
I20260619 08:14:53.328866 10775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:53.336313 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10775
I20260619 08:14:53.336407 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:14:53.336678 2527 external_mini_cluster.cc:1428] Setting key 18ead1934a9318914078a713d25f0e97
I20260619 08:14:53.339322 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.329741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58171 (local address 127.2.119.254:39287)
0619 08:14:53.329812 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:53.329814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:53.330035 (+ 221us) server_negotiation.cc:407] Connection header received
0619 08:14:53.330556 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:53.330557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:53.330566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:53.330635 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:53.331672 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.332180 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.333289 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.333556 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.333615 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.335771 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:53.335772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:53.335772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:53.335779 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:53.337615 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.338135 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.338136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.338137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.338196 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.338507 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.338508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.338508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.338642 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:53.338712 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:53.339069 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 08:14:53.339191 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:14:53.339449 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.329724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:58171)
0619 08:14:53.329896 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:53.329943 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:53.330467 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:53.330695 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:53.330699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:53.331334 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:53.331574 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.331579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.332281 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.332282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.333208 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.333211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.333644 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.333645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.333757 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.334262 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:53.334271 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:53.335665 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:53.337698 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.337706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.337709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.338036 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.338271 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.338272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.338273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.338429 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.338709 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:53.338715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:53.338891 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 08:14:53.339026 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 08:14:53.339239 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260619 08:14:53.340124 10906 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39287
I20260619 08:14:53.340260 10906 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:53.340466 10906 heartbeater.cc:511] Master 127.2.119.254:39287 requested a full tablet report, sending...
I20260619 08:14:53.341051 10712 ts_manager.cc:194] Registered new tserver with Master: fdc25072a20c4147a5b5624598924e2f (127.2.119.193:34611)
I20260619 08:14:53.342054 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:58171
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:14:53.364306 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:53.445513 10911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.445715 10911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.445739 10911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.447013 10911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.447052 10911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.447077 10911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:53.448596 10911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.448848 10911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.449086 10911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.454066 10916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.454130 10911 server_base.cc:1034] running on GCE node
W20260619 08:14:53.454092 10919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.454094 10917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.454543 10911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.454839 10911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.456020 10911 hybrid_clock.cc:648] HybridClock initialized: now 1781856893456004 us; error 27 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.458833 10911 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:14:53.459235 10911 webserver.cc:460] Webserver started at http://127.2.119.194:36601/ using document root <none> and password file <none>
I20260619 08:14:53.459486 10911 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.459535 10911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.459678 10911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:53.460819 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "400da192fb44499faaf8e6be39d84d28"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "a7bb7257b02da48d883761d69fe16499"
server_key_iv: "60bf12a936421ffc08dfad41d028b2f3"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.461202 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "400da192fb44499faaf8e6be39d84d28"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "a7bb7257b02da48d883761d69fe16499"
server_key_iv: "60bf12a936421ffc08dfad41d028b2f3"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.462720 10911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:53.463560 10926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.463771 10911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:14:53.463886 10911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "400da192fb44499faaf8e6be39d84d28"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "a7bb7257b02da48d883761d69fe16499"
server_key_iv: "60bf12a936421ffc08dfad41d028b2f3"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.463968 10911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.474946 10911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.477538 10911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.477718 10911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.477960 10911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:53.478302 10911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:53.478361 10911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.478406 10911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:53.478437 10911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.484664 10911 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:37987
I20260619 08:14:53.484748 11039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:37987 every 8 connection(s)
I20260619 08:14:53.485092 10911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:14:53.489393 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 10911
I20260619 08:14:53.489579 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:14:53.489784 2527 external_mini_cluster.cc:1428] Setting key 8d91587d9a078ea7a21d4bfcb5cb4eb3
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:53.495947 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.485951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:44207 (local address 127.2.119.254:39287)
0619 08:14:53.486008 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:53.486010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:53.486210 (+ 200us) server_negotiation.cc:407] Connection header received
0619 08:14:53.486708 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:53.486709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:53.486719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:53.486783 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:53.487756 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.488322 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.489553 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.489892 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.490084 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.492722 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:53.492724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:53.492724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:53.492733 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:53.494420 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.494864 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.494866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.494867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.494948 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.495225 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.495227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.495227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.495353 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:53.495431 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:53.495750 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 08:14:53.495836 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:14:53.496153 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.485915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:44207)
0619 08:14:53.486069 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:53.486114 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:53.486619 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:53.486846 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:53.486849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:53.487444 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:53.487655 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.487661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.488418 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.488419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.489442 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.489445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.490093 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.490094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.490261 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.490897 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:53.490911 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:53.492615 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:53.494511 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.494518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.494520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.494763 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.495021 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.495022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.495023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.495146 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.495440 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:53.495447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:53.495595 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 08:14:53.495709 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 08:14:53.495883 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260619 08:14:53.496681 11040 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39287
I20260619 08:14:53.496767 11040 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:53.496984 11040 heartbeater.cc:511] Master 127.2.119.254:39287 requested a full tablet report, sending...
I20260619 08:14:53.497457 10712 ts_manager.cc:194] Registered new tserver with Master: 400da192fb44499faaf8e6be39d84d28 (127.2.119.194:37987)
I20260619 08:14:53.497882 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:44207
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:14:53.516597 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:53.601631 11045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.601868 11045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.601908 11045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.603129 11045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.603201 11045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.603247 11045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:53.604740 11045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.605021 11045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.605250 11045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.610376 11051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.610368 11053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.610373 11050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.610572 11045 server_base.cc:1034] running on GCE node
I20260619 08:14:53.610819 11045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.611136 11045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.612313 11045 hybrid_clock.cc:648] HybridClock initialized: now 1781856893612298 us; error 43 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.615079 11045 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:14:53.615545 11045 webserver.cc:460] Webserver started at http://127.2.119.195:44907/ using document root <none> and password file <none>
I20260619 08:14:53.615785 11045 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.615854 11045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.616004 11045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:14:53.617053 11045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "f90ddefc32f44b62ab063ffd5aaa3e5b"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "5ee76b80520cb31f15491aad8c56426c"
server_key_iv: "cde97f5bfa1570fa89a77897a3a85484"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.617450 11045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "f90ddefc32f44b62ab063ffd5aaa3e5b"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "5ee76b80520cb31f15491aad8c56426c"
server_key_iv: "cde97f5bfa1570fa89a77897a3a85484"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.618795 11045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.619719 11060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.619985 11045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.620086 11045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "f90ddefc32f44b62ab063ffd5aaa3e5b"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "5ee76b80520cb31f15491aad8c56426c"
server_key_iv: "cde97f5bfa1570fa89a77897a3a85484"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.620182 11045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.637256 11045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.640040 11045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.640232 11045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.640494 11045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:53.640837 11045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:53.640890 11045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.640931 11045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:53.640959 11045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.646984 11045 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:41037
I20260619 08:14:53.647070 11173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:41037 every 8 connection(s)
I20260619 08:14:53.647490 11045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:14:53.651660 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11045
I20260619 08:14:53.651774 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:14:53.651966 2527 external_mini_cluster.cc:1428] Setting key 74cd41aa782699353f633087a67c6846
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:14:53.657760 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.648411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:49977 (local address 127.2.119.254:39287)
0619 08:14:53.648460 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:53.648462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:53.648695 (+ 233us) server_negotiation.cc:407] Connection header received
0619 08:14:53.649154 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:53.649155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:53.649165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:53.649218 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:53.650174 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.650790 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.652065 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.652364 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.652461 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.654778 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:53.654779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:53.654779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:53.654787 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:53.656295 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.656649 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.656650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.656650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.656703 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:53.657125 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:53.657127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:53.657127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:53.657249 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:53.657387 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:53.657630 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 08:14:53.657696 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:14:53.657922 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:53.648404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:49977)
0619 08:14:53.648557 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:53.648592 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:53.649064 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:53.649271 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:53.649274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:53.649891 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:53.650087 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.650091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.650874 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.650874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.651944 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:53.651947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:53.652468 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:53.652469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:53.652639 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:53.653220 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:53.653227 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:53.654680 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:14:53.656364 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.656369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.656370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.656567 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.656765 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:14:53.656765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:14:53.656766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:14:53.656860 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:14:53.657383 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:14:53.657387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:14:53.657509 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 08:14:53.657597 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:14:53.657720 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260619 08:14:53.658424 11174 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39287
I20260619 08:14:53.658531 11174 heartbeater.cc:464] Registering TS with master...
I20260619 08:14:53.658706 11174 heartbeater.cc:511] Master 127.2.119.254:39287 requested a full tablet report, sending...
I20260619 08:14:53.659093 10712 ts_manager.cc:194] Registered new tserver with Master: f90ddefc32f44b62ab063ffd5aaa3e5b (127.2.119.195:41037)
I20260619 08:14:53.659456 10712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:49977
I20260619 08:14:53.663681 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
Entry for principal oryx/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab.
WARNING: no policy specified for oryx/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
Entry for principal oryx/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab.
WARNING: no policy specified for oryx/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
Entry for principal oryx/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab.
I20260619 08:14:53.736230 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10775
I20260619 08:14:53.740684 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10911
I20260619 08:14:53.745020 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11045
I20260619 08:14:53.749176 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 10680
I20260619 08:14:53.754287 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39287
--webserver_interface=127.2.119.254
--webserver_port=46531
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:39287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:53.835561 11183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.835773 11183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.835796 11183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.836966 11183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:14:53.837006 11183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.837021 11183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.837033 11183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:14:53.837087 11183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:14:53.838596 11183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:39287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=46531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.838852 11183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.839087 11183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.844065 11188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.844077 11189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.844084 11191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.844424 11183 server_base.cc:1034] running on GCE node
I20260619 08:14:53.844705 11183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.844997 11183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.846174 11183 hybrid_clock.cc:648] HybridClock initialized: now 1781856893846152 us; error 37 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.848853 11183 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:14:53.849334 11183 webserver.cc:460] Webserver started at http://127.2.119.254:46531/ using document root <none> and password file <none>
I20260619 08:14:53.849591 11183 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.849674 11183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.851114 11183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.851886 11198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.852144 11183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.852258 11183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "fb8fa01608564539bb14b012fc1798f9"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "142d1056c911ca1c0943727778a3cb14"
server_key_iv: "aa38011956473133091d6b940ee0078d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.852659 11183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.865064 11183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.867456 11183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.867635 11183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.871767 11183 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:39287
I20260619 08:14:53.871814 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:39287 every 8 connection(s)
I20260619 08:14:53.872201 11183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:14:53.873788 11251 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:14:53.874472 11251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Bootstrap starting.
I20260619 08:14:53.878806 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11183
I20260619 08:14:53.879128 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:34611
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:39287
--webserver_port=42425
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:14:53.883311 11251 log.cc:824] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:14:53.885058 11251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:14:53.885427 11251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Bootstrap complete.
I20260619 08:14:53.887730 11251 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.887984 11251 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8fa01608564539bb14b012fc1798f9, State: Initialized, Role: FOLLOWER
I20260619 08:14:53.888202 11251 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.888302 11251 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:14:53.888358 11251 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:14:53.888408 11251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:14:53.889473 11251 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.889591 11251 leader_election.cc:302] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8fa01608564539bb14b012fc1798f9; no voters:
I20260619 08:14:53.889781 11251 leader_election.cc:288] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:14:53.889817 11255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:14:53.889927 11255 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [term 2 LEADER]: Becoming Leader. State: Replica: fb8fa01608564539bb14b012fc1798f9, State: Running, Role: LEADER
I20260619 08:14:53.890009 11255 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } }
I20260619 08:14:53.890058 11251 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:14:53.890271 11255 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8fa01608564539bb14b012fc1798f9. Latest consensus state: current_term: 2 leader_uuid: "fb8fa01608564539bb14b012fc1798f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } } }
I20260619 08:14:53.890419 11255 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:53.890255 11256 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb8fa01608564539bb14b012fc1798f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8fa01608564539bb14b012fc1798f9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39287 } } }
I20260619 08:14:53.890618 11256 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:14:53.890740 11259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:14:53.891228 11259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:14:53.891359 11259 catalog_manager.cc:1259] Loaded cluster ID: a6a9fd7b04bd4fe89d9464c0cb93bec9
I20260619 08:14:53.891394 11259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:14:53.893219 11259 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:14:53.893455 11259 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb8fa01608564539bb14b012fc1798f9: Loaded TSK: 0
I20260619 08:14:53.893673 11259 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:14:53.962468 11253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:53.962702 11253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:53.962726 11253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:53.963900 11253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:53.963940 11253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:53.963966 11253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:14:53.965617 11253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.193.keytab
--principal=oryx/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:34611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=42425
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:53.965855 11253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:53.966086 11253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:53.972014 11277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:53.972054 11278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.972203 11253 server_base.cc:1034] running on GCE node
W20260619 08:14:53.972002 11280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:53.972528 11253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:53.972816 11253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:53.973940 11253 hybrid_clock.cc:648] HybridClock initialized: now 1781856893973916 us; error 39 us; skew 500 ppm
Jun 19 08:14:53 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856893, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:53.977195 11253 init.cc:377] Logged in from keytab as oryx/127.2.119.193@KRBTEST.COM (short username oryx)
I20260619 08:14:53.977746 11253 webserver.cc:460] Webserver started at http://127.2.119.193:42425/ using document root <none> and password file <none>
I20260619 08:14:53.978017 11253 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:53.978077 11253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:53.979489 11253 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.980177 11287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.980363 11253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:53.980463 11253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "fdc25072a20c4147a5b5624598924e2f"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "32c0fbb960b932bb6a528d39f87524bd"
server_key_iv: "c34157e1b075afc894cca848d7e7003d"
server_key_version: "encryptionkey@0"
I20260619 08:14:53.980824 11253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:53.992599 11253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:53.994773 11253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:53.994942 11253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:53.995179 11253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:53.995476 11253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:53.995529 11253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:53.995571 11253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:53.995599 11253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.001986 11253 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34611
I20260619 08:14:54.002070 11402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34611 every 8 connection(s)
I20260619 08:14:54.002393 11253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:14:54.004558 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11253
I20260619 08:14:54.004869 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:37987
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:39287
--webserver_port=36601
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.013438 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.003112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:33399)
0619 08:14:54.003479 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.003519 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.004153 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.004743 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.004747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.005534 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.005797 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.005803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.007071 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.007072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.008429 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.008432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.009267 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.009269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.009475 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.010084 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.010096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.013118 (+ 3022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":186,"threads_started":1}
W20260619 08:14:54.013870 11405 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.003158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33399 (local address 127.2.119.254:39287)
0619 08:14:54.003342 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.003350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.003651 (+ 301us) server_negotiation.cc:407] Connection header received
0619 08:14:54.004383 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.004391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.004468 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.004632 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.005936 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.006957 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.008527 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.009162 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.009270 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.013325 (+ 4055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:33399: BlockingRecv error: recv got EOF from 127.2.119.193:33399 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260619 08:14:54.013903 11403 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:39287 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.119.254:39287: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.019717 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.014217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:48157)
0619 08:14:54.014354 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.014365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.014413 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.014599 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.014601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.014766 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.014831 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.014833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.015797 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.015798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.016696 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.016699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.017140 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.017141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.017210 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.017734 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.017739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.019645 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121}
W20260619 08:14:54.020397 11405 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.014291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:48157 (local address 127.2.119.254:39287)
0619 08:14:54.014323 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.014326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.014471 (+ 145us) server_negotiation.cc:407] Connection header received
0619 08:14:54.014496 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.014497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.014511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.014575 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.014947 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.015705 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.016794 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.017042 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.017132 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.019749 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:48157: BlockingRecv error: recv got EOF from 127.2.119.193:48157 (error 108)
Metrics: {"server-negotiator.queue_time_us":11,"thread_start_us":106,"threads_started":1}
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.025342 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.020134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:47647)
0619 08:14:54.020236 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.020246 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.020286 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.020467 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.020468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.020605 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.020636 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.020637 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.021580 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.021581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.022358 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.022361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.022833 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.022834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.022884 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.023372 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.023377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.025241 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260619 08:14:54.025457 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.020196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:47647 (local address 127.2.119.254:39287)
0619 08:14:54.020243 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.020245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.020275 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:14:54.020333 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.020334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.020345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.020422 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.020716 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.021503 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.022442 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.022750 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.022817 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.025378 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:47647: BlockingRecv error: recv got EOF from 127.2.119.193:47647 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:14:54.025545 11403 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:14:54.091689 11407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:54.091894 11407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:54.091955 11407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:54.093139 11407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:54.093182 11407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:54.093209 11407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:14:54.094667 11407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.194.keytab
--principal=oryx/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:37987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=36601
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:54.095028 11407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:54.095273 11407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:54.100330 11414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:54.100337 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:54.100337 11416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:54.100716 11407 server_base.cc:1034] running on GCE node
I20260619 08:14:54.100921 11407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:54.101178 11407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:54.102393 11407 hybrid_clock.cc:648] HybridClock initialized: now 1781856894102374 us; error 27 us; skew 500 ppm
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856894, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:54.105070 11407 init.cc:377] Logged in from keytab as oryx/127.2.119.194@KRBTEST.COM (short username oryx)
I20260619 08:14:54.105564 11407 webserver.cc:460] Webserver started at http://127.2.119.194:36601/ using document root <none> and password file <none>
I20260619 08:14:54.105762 11407 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:54.105814 11407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:54.107175 11407 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:54.107833 11423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.108054 11407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:54.108157 11407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "400da192fb44499faaf8e6be39d84d28"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "a7bb7257b02da48d883761d69fe16499"
server_key_iv: "60bf12a936421ffc08dfad41d028b2f3"
server_key_version: "encryptionkey@0"
I20260619 08:14:54.108485 11407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:54.124022 11407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:54.126374 11407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:54.126487 11407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:54.126667 11407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:54.126978 11407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:54.127017 11407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.127040 11407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:54.127087 11407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.133070 11407 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:37987
I20260619 08:14:54.133126 11536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:37987 every 8 connection(s)
I20260619 08:14:54.133527 11407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.140928 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.134238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:58033)
0619 08:14:54.134405 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.134441 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.134911 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.135141 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.135145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.135704 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.135898 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.135902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.136650 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.136651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.137583 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.137586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.137958 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.137959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.138055 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.138641 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.138650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.140774 (+ 2124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":106,"threads_started":1}
W20260619 08:14:54.141094 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.134256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:58033 (local address 127.2.119.254:39287)
0619 08:14:54.134353 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.134356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.134538 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:14:54.135014 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.135015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.135029 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.135084 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.136036 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.136558 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.137675 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.137871 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.137939 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.141025 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:58033: BlockingRecv error: recv got EOF from 127.2.119.194:58033 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:14:54.141099 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11407
I20260619 08:14:54.141463 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:41037
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:39287
--webserver_port=44907
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:44537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:14:54.141549 11537 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:39287 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.119.254:39287: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.146165 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.141771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:50181)
0619 08:14:54.141838 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.141849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.141900 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.142045 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.142047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.142208 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.142275 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.142276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.142912 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.142912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.143440 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.143442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.143841 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.143841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.143913 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.144365 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.144371 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.146114 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 08:14:54.146301 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.141794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:50181 (local address 127.2.119.254:39287)
0619 08:14:54.141856 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.141857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.141887 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:14:54.141959 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.141960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.141969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.142018 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.142344 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.142823 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.143543 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.143757 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.143841 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.146229 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:50181: BlockingRecv error: recv got EOF from 127.2.119.194:50181 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.151113 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.146426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:59189)
0619 08:14:54.146533 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.146542 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.146577 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.146704 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.146705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.146806 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.146832 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.146833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.147772 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.147773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.148233 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.148234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.148727 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.148728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.148763 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.149234 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.149238 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.151032 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260619 08:14:54.151192 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.146486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:59189 (local address 127.2.119.254:39287)
0619 08:14:54.146559 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.146561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.146570 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:54.146627 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.146628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.146640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.146704 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.146897 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.147694 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.148311 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.148646 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.148718 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.151128 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:59189: BlockingRecv error: recv got EOF from 127.2.119.194:59189 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 08:14:54.151232 11537 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:14:54.226596 11540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:14:54.226828 11540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:14:54.226866 11540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:14:54.228049 11540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:14:54.228113 11540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:14:54.228158 11540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:14:54.229662 11540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:44537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/krb5kdc/oryx_127.2.119.195.keytab
--principal=oryx/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:41037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=44907
--tserver_master_addrs=127.2.119.254:39287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:14:54.229929 11540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:14:54.230180 11540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:14:54.235401 11545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:54.235401 11548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:14:54.235401 11546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:14:54.235787 11540 server_base.cc:1034] running on GCE node
I20260619 08:14:54.236075 11540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:14:54.236340 11540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:14:54.237504 11540 hybrid_clock.cc:648] HybridClock initialized: now 1781856894237482 us; error 39 us; skew 500 ppm
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856894, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:14:54.240116 11540 init.cc:377] Logged in from keytab as oryx/127.2.119.195@KRBTEST.COM (short username oryx)
I20260619 08:14:54.240653 11540 webserver.cc:460] Webserver started at http://127.2.119.195:44907/ using document root <none> and password file <none>
I20260619 08:14:54.240854 11540 fs_manager.cc:359] Metadata directory not provided
I20260619 08:14:54.240907 11540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:14:54.242242 11540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:54.242815 11555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.243150 11540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:14:54.243265 11540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "f90ddefc32f44b62ab063ffd5aaa3e5b"
format_stamp: "Formatted at 2026-06-19 08:14:53 on dist-test-slave-d0b5"
server_key: "5ee76b80520cb31f15491aad8c56426c"
server_key_iv: "cde97f5bfa1570fa89a77897a3a85484"
server_key_version: "encryptionkey@0"
I20260619 08:14:54.243611 11540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:14:54.268476 11540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:14:54.270641 11540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:14:54.270788 11540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:14:54.271034 11540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:14:54.271358 11540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:14:54.271414 11540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.271447 11540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:14:54.271471 11540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:14:54.276877 11540 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:41037
I20260619 08:14:54.276960 11668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:41037 every 8 connection(s)
I20260619 08:14:54.277364 11540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.284492 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.278060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:58845)
0619 08:14:54.278233 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.278274 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.278766 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.278967 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.278970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.279523 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.279722 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.279727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.280360 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.280361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.281152 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.281154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.281496 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.281497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.281598 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.282129 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.282137 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.284352 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
W20260619 08:14:54.284492 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.278093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:58845 (local address 127.2.119.254:39287)
0619 08:14:54.278159 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.278162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.278383 (+ 221us) server_negotiation.cc:407] Connection header received
0619 08:14:54.278847 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.278848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.278856 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.278909 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.279797 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.280282 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.281226 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.281429 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.281480 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.284445 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:58845: BlockingRecv error: recv got EOF from 127.2.119.195:58845 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 08:14:54.284762 11669 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:39287 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.119.254:39287: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
I20260619 08:14:54.286743 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11540
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.290184 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.284981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:41151)
0619 08:14:54.285031 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.285042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.285081 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.285384 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.285385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.285495 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.285560 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.285561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.286166 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.286166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.286657 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.286659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.287161 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.287162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.287227 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.287714 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.287718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.290102 (+ 2384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:14:54.290235 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.285192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:41151 (local address 127.2.119.254:39287)
0619 08:14:54.285241 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.285242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.285250 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:14:54.285276 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.285277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.285285 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.285368 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.285635 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.286081 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.286737 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.287017 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.287103 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.290171 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:41151: BlockingRecv error: recv got EOF from 127.2.119.195:41151 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:54 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:54.295288 11405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.287112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.2.119.254:39287)
0619 08:14:54.287167 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.287170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.287352 (+ 182us) server_negotiation.cc:407] Connection header received
0619 08:14:54.287444 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.287444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.287456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.287498 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.288125 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.288575 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.289596 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.289835 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.289905 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.291418 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:14:54.291423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:14:54.291423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:14:54.291435 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:14:54.293556 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:54.293985 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:54.293988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:54.293989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:54.294091 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:14:54.294334 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:14:54.294335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:14:54.294336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:14:54.294658 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:14:54.294783 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 08:14:54.294940 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:14:54.295030 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:14:54.295497 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:54.290529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:47423)
0619 08:14:54.290621 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:54.290631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:54.290684 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:54.290803 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:54.290803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:54.290944 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:54.290976 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.290978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.291711 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.291712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.292430 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:54.292433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.292916 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.292917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:54.292957 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.293389 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:54.293394 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:54.295421 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 08:14:54.295528 11408 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:54.290573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:47423 (local address 127.2.119.254:39287)
0619 08:14:54.290617 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:54.290618 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:54.290669 (+ 51us) server_negotiation.cc:407] Connection header received
0619 08:14:54.290741 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:54.290742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:54.290750 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:54.290788 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:54.291069 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.291614 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.292516 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:54.292834 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:54.292910 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:54.295481 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:47423: BlockingRecv error: recv got EOF from 127.2.119.195:47423 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 08:14:54.295702 11669 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 08:14:55.031020 11674 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:55.026032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33317 (local address 127.2.119.254:39287)
0619 08:14:55.026174 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:55.026175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:55.026184 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:55.026250 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:55.026250 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:55.026260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:55.026312 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:55.026675 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.027192 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.027935 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.028195 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.028268 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.030957 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:33317: BlockingRecv error: recv got EOF from 127.2.119.193:33317 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260619 08:14:55.031047 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:55.025929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:33317)
0619 08:14:55.026121 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:55.026132 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:55.026187 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:55.026341 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:55.026342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:55.026505 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:55.026574 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.026576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.027273 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.027274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.027832 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.027834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.028278 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.028279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.028320 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.028771 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:55.028775 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:55.030879 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:55.156594 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:55.151664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:45591)
0619 08:14:55.151797 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:55.151810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:55.151877 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:55.152062 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:55.152063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:55.152243 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:55.152290 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.152292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.152967 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.152968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.153596 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.153598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.153973 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.153974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.154030 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.154540 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:55.154545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:55.156519 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
W20260619 08:14:55.156705 11674 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:55.151735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:45591 (local address 127.2.119.254:39287)
0619 08:14:55.151802 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:55.151805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:55.151857 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:14:55.151931 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:55.151932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:55.151945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:55.152051 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:55.152376 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.152877 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.153674 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.153881 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.153972 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.156636 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:45591: BlockingRecv error: recv got EOF from 127.2.119.194:45591 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:55 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:55.300938 11676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:55.296072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:38999)
0619 08:14:55.296245 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:55.296257 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:55.296343 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:55.296520 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:55.296521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:55.296697 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:55.296745 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.296746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.297513 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.297514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.298265 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:55.298267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.298686 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.298687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:55.298740 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.299176 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:55.299180 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:55.300873 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
W20260619 08:14:55.301061 11674 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:55.296164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:38999 (local address 127.2.119.254:39287)
0619 08:14:55.296245 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:55.296247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:55.296325 (+ 78us) server_negotiation.cc:407] Connection header received
0619 08:14:55.296417 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:55.296418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:55.296431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:55.296513 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:55.296829 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.297416 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.298359 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:55.298612 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:55.298690 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:55.300960 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:38999: BlockingRecv error: recv got EOF from 127.2.119.195:38999 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:56.037091 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:56.031597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:60479)
0619 08:14:56.031735 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:56.031749 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:56.031806 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:56.032047 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:56.032049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:56.032287 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:56.032340 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.032343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.033164 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.033165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.033850 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.033853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.034272 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.034273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.034323 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.034789 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:56.034794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:56.037008 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
W20260619 08:14:56.037256 11678 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:56.031732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:60479 (local address 127.2.119.254:39287)
0619 08:14:56.031857 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:56.031859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:56.031868 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:56.031902 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:56.031903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:56.031914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:56.032009 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:56.032463 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.033060 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.033959 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.034176 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.034246 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.037155 (+ 2909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:60479: BlockingRecv error: recv got EOF from 127.2.119.193:60479 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:56.162168 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:56.157142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:50941)
0619 08:14:56.157352 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:56.157364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:56.157455 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:56.157619 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:56.157620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:56.157807 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:56.157854 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.157855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.158541 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.158542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.159071 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.159081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.159460 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.159461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.159521 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.160079 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:56.160084 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:56.162071 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":98,"threads_started":1}
W20260619 08:14:56.162225 11678 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:56.157236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:50941 (local address 127.2.119.254:39287)
0619 08:14:56.157339 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:56.157342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:56.157436 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:14:56.157531 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:56.157531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:56.157546 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:56.157609 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:56.157935 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.158453 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.159169 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.159373 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.159459 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.162160 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:50941: BlockingRecv error: recv got EOF from 127.2.119.194:50941 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:56 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:56.305806 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:56.301399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:46991)
0619 08:14:56.301527 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:56.301537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:56.301586 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:56.301748 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:56.301749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:56.301898 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:56.301948 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.301950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.302570 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.302571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.303138 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:56.303140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.303512 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.303513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:56.303557 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.303988 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:56.303992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:56.305749 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
W20260619 08:14:56.305945 11678 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:56.301501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:46991 (local address 127.2.119.254:39287)
0619 08:14:56.301561 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:56.301563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:56.301577 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:14:56.301656 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:56.301657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:56.301670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:56.301732 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:56.302011 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.302502 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.303245 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:56.303435 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:56.303502 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:56.305848 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:46991: BlockingRecv error: recv got EOF from 127.2.119.195:46991 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:57.042533 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:57.037674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:33483)
0619 08:14:57.037814 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:57.037830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:57.037880 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:57.038084 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:57.038085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:57.038274 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:57.038325 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.038327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.038991 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.038992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.039550 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.039552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.039932 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.039933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.039989 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.040472 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:57.040478 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:57.042451 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
W20260619 08:14:57.042798 11682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:57.037779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33483 (local address 127.2.119.254:39287)
0619 08:14:57.037921 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:57.037923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:57.037930 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:14:57.037951 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:57.037952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:57.037963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:57.038034 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:57.038404 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.038918 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.039643 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.039831 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.039884 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.042592 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:33483: BlockingRecv error: recv got EOF from 127.2.119.193:33483 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:57.167671 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:57.162691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:34821)
0619 08:14:57.162856 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:57.162865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:57.162923 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:57.163063 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:57.163064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:57.163243 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:57.163290 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.163292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.164014 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.164015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.164582 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.164583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.165083 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.165084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.165159 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.165820 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:57.165824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:57.167613 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
W20260619 08:14:57.167814 11682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:57.162773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:34821 (local address 127.2.119.254:39287)
0619 08:14:57.162865 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:57.162872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:57.162960 (+ 88us) server_negotiation.cc:407] Connection header received
0619 08:14:57.162986 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:57.162987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:57.162998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:57.163059 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:57.163374 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.163870 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.164676 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.164882 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.164993 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.167733 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:34821: BlockingRecv error: recv got EOF from 127.2.119.194:34821 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:57 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:57.311216 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:57.306341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:35947)
0619 08:14:57.306478 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:57.306496 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:57.306555 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:57.306706 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:57.306707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:57.306864 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:57.306914 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.306916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.307571 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.307572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.308188 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:57.308190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.308641 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.308642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:57.308708 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.309205 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:57.309215 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:57.311129 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
W20260619 08:14:57.311293 11682 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:57.306396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:35947 (local address 127.2.119.254:39287)
0619 08:14:57.306473 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:57.306475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:57.306556 (+ 81us) server_negotiation.cc:407] Connection header received
0619 08:14:57.306600 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:57.306601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:57.306614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:57.306681 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:57.306978 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.307478 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.308302 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:57.308539 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:57.308639 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:57.311230 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:35947: BlockingRecv error: recv got EOF from 127.2.119.195:35947 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:58.048209 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:58.043124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:46269)
0619 08:14:58.043288 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:58.043299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:58.043355 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:58.043496 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:58.043497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:58.043672 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:58.043722 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.043723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.044495 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.044496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.045034 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.045036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.045488 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.045489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.045563 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.046243 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:58.046248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:58.048149 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
W20260619 08:14:58.048336 11686 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:58.043227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46269 (local address 127.2.119.254:39287)
0619 08:14:58.043363 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:58.043365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:58.043374 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:58.043397 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:58.043398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:58.043412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:58.043480 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:58.043824 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.044343 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.045126 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.045403 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.045488 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.048250 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:46269: BlockingRecv error: recv got EOF from 127.2.119.193:46269 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:58.177552 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:58.168207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:44521)
0619 08:14:58.168355 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:58.168367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:58.168431 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:58.168588 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:58.168588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:58.168757 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:58.168811 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.168813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.169565 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.169566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.170141 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.170143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.170485 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.170486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.170539 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.175625 (+ 5086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:58.175630 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:58.177493 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260619 08:14:58.177667 11686 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:58.168324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:44521 (local address 127.2.119.254:39287)
0619 08:14:58.168391 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:58.168393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:58.168415 (+ 22us) server_negotiation.cc:407] Connection header received
0619 08:14:58.168495 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:58.168495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:58.168508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:58.168572 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:58.168909 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.169486 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.170216 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.170403 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.170457 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.177591 (+ 7134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:44521: BlockingRecv error: recv got EOF from 127.2.119.194:44521 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:58 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:58.316979 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:58.311847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:42087)
0619 08:14:58.312070 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:58.312084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:58.312195 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:58.312380 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:58.312381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:58.312574 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:58.312621 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.312622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.313337 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.313338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.313881 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:58.313883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.314265 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.314266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:58.314322 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.314882 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:58.314887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:58.316883 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":86,"threads_started":1}
W20260619 08:14:58.317027 11686 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:58.311951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:42087 (local address 127.2.119.254:39287)
0619 08:14:58.312074 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:58.312077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:58.312174 (+ 97us) server_negotiation.cc:407] Connection header received
0619 08:14:58.312305 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:58.312306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:58.312323 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:58.312376 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:58.312712 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.313227 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.313995 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:58.314188 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:58.314268 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:58.316964 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:42087: BlockingRecv error: recv got EOF from 127.2.119.195:42087 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:59.053589 11689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:59.048732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:58851)
0619 08:14:59.048889 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:59.048911 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:59.049033 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:59.049202 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:59.049202 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:59.049397 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:59.049453 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.049455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.050141 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.050142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.050719 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.050720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.051083 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.051083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.051131 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.051575 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:59.051580 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:59.053523 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
W20260619 08:14:59.053757 11690 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:59.048853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58851 (local address 127.2.119.254:39287)
0619 08:14:59.048981 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:59.048983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:59.049014 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:14:59.049100 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:59.049101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:59.049114 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:59.049178 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:59.049524 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.050050 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.050807 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.051000 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.051060 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.053649 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:58851: BlockingRecv error: recv got EOF from 127.2.119.193:58851 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:59.183701 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:59.178104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:51591)
0619 08:14:59.178284 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:59.178300 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:59.178348 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:59.178568 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:59.178568 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:59.178731 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:59.178783 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.178784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.180027 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.180028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.180574 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.180577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.180951 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.180952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.180993 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.181568 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:59.181573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:59.183622 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
W20260619 08:14:59.183820 11690 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:59.178326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:51591 (local address 127.2.119.254:39287)
0619 08:14:59.178412 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:59.178414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:59.178426 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:14:59.178457 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:59.178457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:59.178469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:59.178538 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:59.178913 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.179952 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.180678 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.180873 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.180936 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.183719 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:51591: BlockingRecv error: recv got EOF from 127.2.119.194:51591 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:14:59 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:14:59.322224 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:14:59.317529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:38133)
0619 08:14:59.317681 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:14:59.317693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:14:59.317754 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:14:59.317914 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:14:59.317915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:14:59.318071 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:14:59.318118 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.318120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.318847 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.318848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.319439 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:14:59.319441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.319842 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.319843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:14:59.319895 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.320365 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:14:59.320370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:14:59.322166 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
W20260619 08:14:59.322342 11690 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:14:59.317716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:38133 (local address 127.2.119.254:39287)
0619 08:14:59.317781 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:14:59.317783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:14:59.317792 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:14:59.317820 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:14:59.317821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:14:59.317833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:14:59.317888 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:14:59.318242 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.318749 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.319552 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:14:59.319749 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:14:59.319816 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:14:59.322264 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:38133: BlockingRecv error: recv got EOF from 127.2.119.195:38133 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:00.059170 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:00.054220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:36839)
0619 08:15:00.054401 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:00.054413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:00.054470 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:00.054645 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:00.054646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:00.054892 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:00.054969 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.054971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.055667 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.055668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.056256 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.056259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.056658 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.056659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.056714 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.057277 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:00.057287 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:00.059108 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
W20260619 08:15:00.059347 11694 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:00.054313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:36839 (local address 127.2.119.254:39287)
0619 08:15:00.054498 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:00.054500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:00.054508 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:00.054533 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:00.054534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:00.054548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:00.054619 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:00.055052 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.055589 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.056350 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.056563 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.056659 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.059263 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:36839: BlockingRecv error: recv got EOF from 127.2.119.193:36839 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:00.189283 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:00.184271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:33137)
0619 08:15:00.184404 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:00.184417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:00.184488 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:00.184634 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:00.184635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:00.184816 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:00.184903 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.184905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.185856 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.185857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.186431 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.186433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.186769 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.186769 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.186810 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.187254 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:00.187258 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:00.189224 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
W20260619 08:15:00.189507 11694 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:00.184366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:33137 (local address 127.2.119.254:39287)
0619 08:15:00.184441 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:00.184443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:00.184472 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:15:00.184548 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:00.184549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:00.184559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:00.184610 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:00.184995 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.185779 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.186504 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.186700 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.186752 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.189435 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:33137: BlockingRecv error: recv got EOF from 127.2.119.194:33137 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:00 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:00.327497 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:00.322750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:36547)
0619 08:15:00.322911 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:00.322922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:00.322988 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:00.323127 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:00.323127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:00.323290 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:00.323337 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.323339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.324042 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.324042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.324587 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:00.324589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.324947 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.324948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:00.324989 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.325472 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:00.325476 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:00.327431 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
W20260619 08:15:00.327749 11694 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:00.322834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:36547 (local address 127.2.119.254:39287)
0619 08:15:00.322922 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:00.322928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:00.322974 (+ 46us) server_negotiation.cc:407] Connection header received
0619 08:15:00.323046 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:00.323047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:00.323060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:00.323104 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:00.323405 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.323963 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.324688 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:00.324878 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:00.324933 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:00.327665 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:36547: BlockingRecv error: recv got EOF from 127.2.119.195:36547 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:01.064627 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:01.059653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:58155)
0619 08:15:01.059808 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:01.059825 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:01.059883 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:01.060145 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:01.060146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:01.060304 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:01.060357 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.060359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.061097 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.061098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.061639 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.061641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.061974 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.061975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.062014 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.062567 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:01.062572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:01.064555 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260619 08:15:01.064766 11698 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:01.059852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58155 (local address 127.2.119.254:39287)
0619 08:15:01.059986 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:01.059992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:01.060000 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:01.060022 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:01.060023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:01.060033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:01.060100 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:01.060487 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.061011 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.061711 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.061906 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.061961 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.064663 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:58155: BlockingRecv error: recv got EOF from 127.2.119.193:58155 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:01.195278 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:01.189912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:44919)
0619 08:15:01.190118 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:01.190146 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:01.190233 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:01.190730 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:01.190732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:01.190909 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:01.190956 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.190958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.191811 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.191812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.192361 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.192363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.192782 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.192782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.192841 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.193368 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:01.193372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:01.195180 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":152,"threads_started":1}
W20260619 08:15:01.195336 11698 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:01.190407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:44919 (local address 127.2.119.254:39287)
0619 08:15:01.190507 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:01.190509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:01.190519 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:15:01.190571 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:01.190572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:01.190631 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:01.190700 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:01.191043 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.191718 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.192467 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.192689 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.192781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.195275 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:44919: BlockingRecv error: recv got EOF from 127.2.119.194:44919 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:01 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:01.333122 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:01.328021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:50943)
0619 08:15:01.328197 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:01.328213 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:01.328270 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:01.328421 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:01.328422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:01.328577 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:01.328627 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.328628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.329524 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.329525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.330056 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:01.330058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.330480 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.330481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:01.330545 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.331055 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:01.331065 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:01.333060 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
W20260619 08:15:01.333338 11698 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:01.328154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:50943 (local address 127.2.119.254:39287)
0619 08:15:01.328218 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:01.328225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:01.328255 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:15:01.328328 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:01.328329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:01.328342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:01.328399 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:01.328722 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.329431 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.330177 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:01.330387 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:01.330477 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:01.333223 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:50943: BlockingRecv error: recv got EOF from 127.2.119.195:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:02.070363 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:02.065172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:33275)
0619 08:15:02.065355 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:02.065367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:02.065428 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:02.065653 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:02.065654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:02.065823 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:02.065881 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.065882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.066589 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.066590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.067165 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.067167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.067557 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.067558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.067597 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.068224 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:02.068234 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:02.070303 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
W20260619 08:15:02.070499 11702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:02.065358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33275 (local address 127.2.119.254:39287)
0619 08:15:02.065510 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:02.065511 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:02.065519 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:02.065545 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:02.065545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:02.065556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:02.065622 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:02.065958 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.066482 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.067269 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.067464 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.067541 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.070416 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:33275: BlockingRecv error: recv got EOF from 127.2.119.193:33275 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:02.201363 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:02.195863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:54557)
0619 08:15:02.196009 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:02.196021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:02.196082 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:02.196215 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:02.196216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:02.196378 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:02.196429 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.196430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.197462 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.197463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.197992 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.197994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.198360 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.198361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.198412 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.198871 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:02.198876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:02.201229 (+ 2353us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
W20260619 08:15:02.201406 11702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:02.195920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:54557 (local address 127.2.119.254:39287)
0619 08:15:02.196013 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:02.196015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:02.196081 (+ 66us) server_negotiation.cc:407] Connection header received
0619 08:15:02.196126 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:02.196127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:02.196138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:02.196200 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:02.196539 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.197255 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.198073 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.198262 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.198320 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.201341 (+ 3021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:54557: BlockingRecv error: recv got EOF from 127.2.119.194:54557 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:02 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:02.338574 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:02.333644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:49401)
0619 08:15:02.333786 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:02.333798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:02.333885 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:02.334021 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:02.334022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:02.334217 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:02.334267 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.334269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.335063 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.335064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.335602 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:02.335604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.335988 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.335989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:02.336060 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.336518 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:02.336522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:02.338517 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
W20260619 08:15:02.338718 11702 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:02.333725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:49401 (local address 127.2.119.254:39287)
0619 08:15:02.333773 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:02.333775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:02.333896 (+ 121us) server_negotiation.cc:407] Connection header received
0619 08:15:02.333926 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:02.333927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:02.333942 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:02.334002 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:02.334378 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.334930 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.335715 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:02.335908 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:02.335972 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:02.338631 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:49401: BlockingRecv error: recv got EOF from 127.2.119.195:49401 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:03.075820 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:03.070861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:46543)
0619 08:15:03.071047 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:03.071062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:03.071143 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:03.071407 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:03.071408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:03.071582 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:03.071632 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.071634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.072391 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.072392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.072921 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.072923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.073263 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.073263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.073320 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.073839 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:03.073844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:03.075760 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":95,"threads_started":1}
W20260619 08:15:03.076099 11706 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:03.071000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46543 (local address 127.2.119.254:39287)
0619 08:15:03.071222 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:03.071224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:03.071234 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:15:03.071256 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:03.071256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:03.071266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:03.071335 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:03.071779 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.072294 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.072999 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.073193 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.073242 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.075944 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:46543: BlockingRecv error: recv got EOF from 127.2.119.193:46543 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:03.210484 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:03.201949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:59627)
0619 08:15:03.202115 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:03.202133 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:03.202186 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:03.202337 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:03.202338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:03.202517 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:03.202577 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.202579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.203362 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.203363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.204065 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.204069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.204642 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.204643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.204737 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.208319 (+ 3582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:03.208326 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:03.210375 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":117,"threads_started":1}
W20260619 08:15:03.210542 11706 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:03.202014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:59627 (local address 127.2.119.254:39287)
0619 08:15:03.202167 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:03.202169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:03.202180 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:15:03.202253 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:03.202254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:03.202268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:03.202317 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:03.202674 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.203265 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.204163 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.204546 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.204632 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.210473 (+ 5841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:59627: BlockingRecv error: recv got EOF from 127.2.119.194:59627 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:03 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:03.344288 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:03.339080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:41063)
0619 08:15:03.339279 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:03.339289 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:03.339337 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:03.339522 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:03.339524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:03.339698 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:03.339757 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.339759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.340502 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.340503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.341069 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:03.341072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.341529 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.341529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:03.341581 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.342160 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:03.342167 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:03.344215 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
W20260619 08:15:03.344470 11706 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:03.339219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:41063 (local address 127.2.119.254:39287)
0619 08:15:03.339314 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:03.339316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:03.339326 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:15:03.339401 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:03.339401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:03.339414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:03.339471 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:03.339872 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.340401 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.341170 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:03.341438 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:03.341506 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:03.344364 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:41063: BlockingRecv error: recv got EOF from 127.2.119.195:41063 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:04.081481 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:04.076426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:57475)
0619 08:15:04.076572 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:04.076585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:04.076656 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:04.076895 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:04.076896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:04.077124 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:04.077173 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.077174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.077878 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.077879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.078524 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.078535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.078904 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.078905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.078957 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.079555 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:04.079561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:04.081421 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260619 08:15:04.081640 11710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:04.076487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:57475 (local address 127.2.119.254:39287)
0619 08:15:04.076604 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:04.076606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:04.076640 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:15:04.076770 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:04.076771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:04.076787 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:04.076862 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:04.077244 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.077774 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.078617 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.078812 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.078864 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.081548 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:57475: BlockingRecv error: recv got EOF from 127.2.119.193:57475 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:04.215906 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:04.211036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:36699)
0619 08:15:04.211176 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:04.211237 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:04.211324 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:04.211507 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:04.211508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:04.211720 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:04.211777 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.211779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.212474 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.212475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.213011 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.213012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.213475 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.213476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.213539 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.213977 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:04.213982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:04.215850 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
W20260619 08:15:04.216038 11710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:04.211085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:36699 (local address 127.2.119.254:39287)
0619 08:15:04.211179 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:04.211182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:04.211303 (+ 121us) server_negotiation.cc:407] Connection header received
0619 08:15:04.211398 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:04.211398 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:04.211411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:04.211485 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:04.211861 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.212384 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.213105 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.213375 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.213473 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.215954 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:36699: BlockingRecv error: recv got EOF from 127.2.119.194:36699 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:04 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:04.349967 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:04.344837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:34371)
0619 08:15:04.344986 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:04.345029 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:04.345090 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:04.345265 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:04.345266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:04.345462 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:04.345509 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.345511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.346277 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.346278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.346878 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:04.346880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.347255 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.347256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:04.347315 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.347797 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:04.347802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:04.349910 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260619 08:15:04.350112 11710 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:04.344982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:34371 (local address 127.2.119.254:39287)
0619 08:15:04.345064 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:04.345066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:04.345075 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:15:04.345165 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:04.345166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:04.345181 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:04.345258 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:04.345599 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.346176 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.346963 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:04.347162 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:04.347252 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:04.350016 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:34371: BlockingRecv error: recv got EOF from 127.2.119.195:34371 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:05.087533 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:05.082073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:34169)
0619 08:15:05.082190 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:05.082207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:05.082257 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:05.082453 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:05.082453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:05.082645 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:05.082694 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.082695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.083541 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.083542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.084098 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.084100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.084740 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.084742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.084815 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.085513 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:05.085518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:05.087461 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
W20260619 08:15:05.087705 11714 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:05.082149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:34169 (local address 127.2.119.254:39287)
0619 08:15:05.082317 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:05.082319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:05.082327 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:05.082350 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:05.082351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:05.082361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:05.082426 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:05.082793 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.083389 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.084216 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.084592 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.084718 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.087619 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:34169: BlockingRecv error: recv got EOF from 127.2.119.193:34169 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:05.221484 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:05.216480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:49283)
0619 08:15:05.216613 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:05.216625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:05.216684 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:05.216799 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:05.216799 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:05.216958 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:05.217063 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.217065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.217827 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.217828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.218416 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.218419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.218879 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.218880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.218927 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.219504 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:05.219509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:05.221415 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
W20260619 08:15:05.221618 11714 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:05.216575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:49283 (local address 127.2.119.254:39287)
0619 08:15:05.216688 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:05.216690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:05.216700 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:15:05.216727 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:05.216728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:05.216738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:05.216787 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:05.217172 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.217729 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.218531 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.218781 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.218847 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.221513 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:49283: BlockingRecv error: recv got EOF from 127.2.119.194:49283 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:05 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:05.355293 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:05.350485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:53467)
0619 08:15:05.350626 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:05.350639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:05.350701 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:05.350864 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:05.350864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:05.351046 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:05.351092 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.351093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.351860 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.351861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.352414 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:05.352416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.352803 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.352804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:05.352868 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.353417 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:05.353422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:05.355188 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260619 08:15:05.355326 11714 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:05.350614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:53467 (local address 127.2.119.254:39287)
0619 08:15:05.350719 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:05.350721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:05.350730 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:15:05.350759 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:05.350759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:05.350773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:05.350857 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:05.351245 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.351767 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.352507 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:05.352709 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:05.352801 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:05.355261 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:53467: BlockingRecv error: recv got EOF from 127.2.119.195:53467 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:06.092907 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:06.088009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:55243)
0619 08:15:06.088175 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:06.088187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:06.088245 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:06.088399 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:06.088400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:06.088564 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:06.088614 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.088615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.089331 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.089332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.089940 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.089943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.090367 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.090368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.090411 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.090962 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:06.090967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:06.092848 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
W20260619 08:15:06.093086 11718 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:06.088100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:55243 (local address 127.2.119.254:39287)
0619 08:15:06.088273 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:06.088274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:06.088282 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:06.088304 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:06.088304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:06.088314 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:06.088379 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:06.088687 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.089220 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.090046 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.090275 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.090372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.092989 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:55243: BlockingRecv error: recv got EOF from 127.2.119.193:55243 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:06.226766 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:06.222074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:56241)
0619 08:15:06.222237 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:06.222250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:06.222310 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:06.222443 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:06.222443 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:06.222608 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:06.222662 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.222664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.223404 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.223405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.223933 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.223935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.224321 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.224322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.224392 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.224860 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:06.224864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:06.226708 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
W20260619 08:15:06.226869 11718 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:06.222214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:56241 (local address 127.2.119.254:39287)
0619 08:15:06.222294 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:06.222296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:06.222304 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:06.222369 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:06.222370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:06.222381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:06.222422 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:06.222760 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.223326 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.224010 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.224225 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.224320 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.226800 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:56241: BlockingRecv error: recv got EOF from 127.2.119.194:56241 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:06 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:06.360689 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:06.355793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:56397)
0619 08:15:06.356055 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:06.356067 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:06.356136 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:06.356315 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:06.356316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:06.356509 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:06.356563 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.356565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.357283 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.357285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.357851 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:06.357853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.358231 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.358232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:06.358292 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.358722 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:06.358727 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:06.360577 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":195,"threads_started":1}
W20260619 08:15:06.360716 11718 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:06.355988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:56397 (local address 127.2.119.254:39287)
0619 08:15:06.356090 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:06.356092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:06.356116 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:15:06.356210 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:06.356211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:06.356227 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:06.356321 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:06.356657 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.357190 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.357942 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:06.358134 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:06.358232 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:06.360657 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:56397: BlockingRecv error: recv got EOF from 127.2.119.195:56397 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:07.098443 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:07.093402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:49265)
0619 08:15:07.093568 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:07.093578 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:07.093634 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:07.093810 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:07.093810 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:07.093984 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:07.094032 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.094033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.094701 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.094702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.095249 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.095251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.095678 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.095679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.095764 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.096380 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:07.096384 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:07.098344 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
W20260619 08:15:07.098503 11722 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:07.093503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:49265 (local address 127.2.119.254:39287)
0619 08:15:07.093681 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:07.093686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:07.093693 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:15:07.093714 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:07.093715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:07.093730 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:07.093790 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:07.094104 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.094623 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.095336 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.095542 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.095607 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.098429 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:49265: BlockingRecv error: recv got EOF from 127.2.119.193:49265 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:07.232009 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:07.227286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:42325)
0619 08:15:07.227400 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:07.227416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:07.227463 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:07.227597 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:07.227599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:07.227765 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:07.227812 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.227814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.228475 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.228476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.229012 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.229013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.229437 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.229438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.229481 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.229935 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:07.229940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:07.231911 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
W20260619 08:15:07.232060 11722 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:07.227396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:42325 (local address 127.2.119.254:39287)
0619 08:15:07.227461 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:07.227463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:07.227501 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:15:07.227525 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:07.227526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:07.227537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:07.227577 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:07.227880 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.228395 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.229098 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.229362 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.229427 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.231994 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:42325: BlockingRecv error: recv got EOF from 127.2.119.194:42325 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:07 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:07.365917 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:07.361353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:51409)
0619 08:15:07.361498 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:07.361515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:07.361579 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:07.361710 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:07.361711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:07.361874 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:07.361922 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.361923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.362569 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.362570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.363094 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:07.363095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.363474 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.363475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:07.363517 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.364020 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:07.364030 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:07.365856 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
W20260619 08:15:07.366081 11722 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:07.361404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:51409 (local address 127.2.119.254:39287)
0619 08:15:07.361506 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:07.361508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:07.361588 (+ 80us) server_negotiation.cc:407] Connection header received
0619 08:15:07.361629 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:07.361630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:07.361644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:07.361686 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:07.361993 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.362497 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.363205 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:07.363398 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:07.363462 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:07.365989 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:51409: BlockingRecv error: recv got EOF from 127.2.119.195:51409 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:08.103996 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:08.098956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:54881)
0619 08:15:08.099129 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:08.099141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:08.099205 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:08.099375 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:08.099376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:08.099562 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:08.099620 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.099621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.100322 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.100323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.100894 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.100896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.101377 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.101378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.101435 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.101933 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:08.101938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:08.103930 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":90,"threads_started":1}
W20260619 08:15:08.104125 11726 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:08.099060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54881 (local address 127.2.119.254:39287)
0619 08:15:08.099220 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:08.099222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:08.099249 (+ 27us) server_negotiation.cc:407] Connection header received
0619 08:15:08.099276 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:08.099277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:08.099288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:08.099339 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:08.099697 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.100226 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.100984 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.101233 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.101314 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.104033 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:54881: BlockingRecv error: recv got EOF from 127.2.119.193:54881 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:08.242482 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:08.232555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:52369)
0619 08:15:08.232718 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:08.232730 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:08.232794 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:08.232973 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:08.232975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:08.233159 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:08.233207 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.233209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.233940 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.233941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.234515 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.234517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.234910 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.234911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.234963 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.240156 (+ 5193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:08.240162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:08.242411 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
W20260619 08:15:08.242575 11726 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:08.232681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:52369 (local address 127.2.119.254:39287)
0619 08:15:08.232741 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:08.232743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:08.232780 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:15:08.232856 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:08.232856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:08.232867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:08.232912 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:08.233314 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.233857 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.234617 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.234817 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.234879 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.242529 (+ 7650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:52369: BlockingRecv error: recv got EOF from 127.2.119.194:52369 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:08 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.195@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:08.371390 11728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:08.366395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.195:43351)
0619 08:15:08.366568 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:08.366586 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:08.366654 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:08.366837 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:08.366838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:08.367005 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:08.367058 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.367059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.367759 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.367761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.368426 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:08.368428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.368807 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.368808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:08.368852 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.369347 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:08.369352 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:08.371262 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260619 08:15:08.371412 11726 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:08.366524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:43351 (local address 127.2.119.254:39287)
0619 08:15:08.366593 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:08.366595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:08.366647 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:15:08.366716 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:08.366716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:08.366731 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:08.366780 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:08.367137 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.367662 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.368524 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:08.368732 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:08.368791 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:08.371348 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.195:43351: BlockingRecv error: recv got EOF from 127.2.119.195:43351 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 08:15:09 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:09 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.193@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:09.109035 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.104596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.193:58453)
0619 08:15:09.104734 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:09.104744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:09.104791 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:09.104931 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:09.104931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:09.105088 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:09.105137 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.105139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.105815 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.105815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.106357 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.106359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.106734 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.106735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.106787 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.107253 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:09.107258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:09.108971 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
W20260619 08:15:09.109112 11730 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:09.104669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58453 (local address 127.2.119.254:39287)
0619 08:15:09.104810 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.104811 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.104819 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:09.104841 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.104841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.104852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.104914 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.105216 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.105748 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.106440 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.106646 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.106701 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.109059 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.193:58453: BlockingRecv error: recv got EOF from 127.2.119.193:58453 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
Jun 19 08:15:09 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 08:15:09 dist-test-slave-d0b5 krb5kdc[10667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.119.194@KRBTEST.COM for oryx/127.2.119.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 08:15:09.248404 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.243395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39287 (local address 127.2.119.194:38545)
0619 08:15:09.243667 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:09.243677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:09.243733 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:09.243894 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:09.243896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:09.244064 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:09.244116 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.244118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.244791 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.244792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.245409 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.245412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.245814 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.245815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.245859 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.246398 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:09.246403 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:09.248344 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.119.254:39287: Server oryx/127.2.119.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":94,"threads_started":1}
W20260619 08:15:09.248582 11730 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:09.243474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:38545 (local address 127.2.119.254:39287)
0619 08:15:09.243537 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.243539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.243725 (+ 186us) server_negotiation.cc:407] Connection header received
0619 08:15:09.243778 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.243779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.243790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.243844 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.244196 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.244700 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.245489 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.245733 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.245787 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.248485 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.119.194:38545: BlockingRecv error: recv got EOF from 127.2.119.194:38545 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:15:09.287258 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11253
I20260619 08:15:09.291955 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11407
I20260619 08:15:09.296077 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11540
I20260619 08:15:09.300244 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11183
2026-06-19T08:15:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17449 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T08:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:15:09Z Disabled control of system clock
I20260619 08:15:09.328344 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:37437
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 08:15:09.409192 11735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:09.409492 11735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:09.409518 11735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:09.410781 11735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:15:09.410826 11735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:09.410840 11735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:09.410852 11735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:15:09.410864 11735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:15:09.412256 11735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:37437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:09.412526 11735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:09.412853 11735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:09.418244 11741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:09.418310 11735 server_base.cc:1034] running on GCE node
W20260619 08:15:09.418352 11740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.418272 11743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:09.418838 11735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:09.419152 11735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:09.420348 11735 hybrid_clock.cc:648] HybridClock initialized: now 1781856909420320 us; error 52 us; skew 500 ppm
I20260619 08:15:09.421798 11735 webserver.cc:460] Webserver started at http://127.2.119.254:39725/ using document root <none> and password file <none>
I20260619 08:15:09.422063 11735 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:09.422120 11735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:09.422294 11735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:09.423415 11735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "685c178b8c3f4e4da9d6dead2b2a0c34"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "1dee1ff409c2db02d609f379de176ea4"
server_key_iv: "1e2b29215906cc99e8e856d69ad311c8"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.423794 11735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "685c178b8c3f4e4da9d6dead2b2a0c34"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "1dee1ff409c2db02d609f379de176ea4"
server_key_iv: "1e2b29215906cc99e8e856d69ad311c8"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.425124 11735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 08:15:09.425976 11749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.426189 11735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:15:09.426302 11735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "685c178b8c3f4e4da9d6dead2b2a0c34"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "1dee1ff409c2db02d609f379de176ea4"
server_key_iv: "1e2b29215906cc99e8e856d69ad311c8"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.426381 11735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:09.442536 11735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:09.442875 11735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:09.443027 11735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:09.446929 11735 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:37437
I20260619 08:15:09.446978 11801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:37437 every 8 connection(s)
I20260619 08:15:09.447343 11735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:15:09.448046 11802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:09.450786 11802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34: Bootstrap starting.
I20260619 08:15:09.451409 11802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:09.451725 11802 log.cc:824] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:09.452410 11802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34: No bootstrap required, opened a new log
I20260619 08:15:09.452831 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11735
I20260619 08:15:09.452934 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:15:09.453130 2527 external_mini_cluster.cc:1428] Setting key 37c435de23e8f128fc23d953f43d448e
I20260619 08:15:09.454632 11802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } }
I20260619 08:15:09.454802 11802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:09.454866 11802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 685c178b8c3f4e4da9d6dead2b2a0c34, State: Initialized, Role: FOLLOWER
I20260619 08:15:09.455070 11802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } }
I20260619 08:15:09.455181 11802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:15:09.455228 11802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:15:09.455284 11802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:09.455850 11802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } }
I20260619 08:15:09.455967 11802 leader_election.cc:302] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 685c178b8c3f4e4da9d6dead2b2a0c34; no voters:
I20260619 08:15:09.456187 11802 leader_election.cc:288] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:15:09.456220 11807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:09.456331 11807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [term 1 LEADER]: Becoming Leader. State: Replica: 685c178b8c3f4e4da9d6dead2b2a0c34, State: Running, Role: LEADER
I20260619 08:15:09.456456 11802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:15:09.456475 11807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } }
I20260619 08:15:09.456900 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 685c178b8c3f4e4da9d6dead2b2a0c34. Latest consensus state: current_term: 1 leader_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } } }
I20260619 08:15:09.457062 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:09.457347 11816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:15:09.457962 11809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "685c178b8c3f4e4da9d6dead2b2a0c34" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37437 } } }
I20260619 08:15:09.458127 11809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:09.457973 11816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:15:09.460063 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.453713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43028 (local address 127.2.119.254:37437)
0619 08:15:09.453940 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.453949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.453982 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:15:09.454605 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.454612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.454688 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.454831 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.455075 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.456641 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.457727 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.458369 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.458518 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.459242 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:09.459246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:09.459246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:09.459454 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:09.459502 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:09.459517 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:09.459586 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:09.459738 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 08:15:09.459825 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260619 08:15:09.461194 11816 catalog_manager.cc:1347] Generated new cluster ID: 466189a2cd7c4b15a3f9cf1ec65ca10e
I20260619 08:15:09.461246 11816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:15:09.476262 11816 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:15:09.476835 11816 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:15:09.487533 11816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 685c178b8c3f4e4da9d6dead2b2a0c34: Generated new TSK 0
I20260619 08:15:09.487774 11816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:15:09.495214 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--builtin_ntp_servers=127.2.119.212:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:15:09.578081 11826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:09.578303 11826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:09.578325 11826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:09.579552 11826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:09.579596 11826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:09.579622 11826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:15:09.581029 11826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:09.581280 11826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:09.581652 11826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:09.587335 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.587335 11832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.587352 11834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:09.587634 11826 server_base.cc:1034] running on GCE node
I20260619 08:15:09.587894 11826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:09.588201 11826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:09.589385 11826 hybrid_clock.cc:648] HybridClock initialized: now 1781856909589364 us; error 43 us; skew 500 ppm
I20260619 08:15:09.590772 11826 webserver.cc:460] Webserver started at http://127.2.119.193:36875/ using document root <none> and password file <none>
I20260619 08:15:09.591039 11826 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:09.591090 11826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:09.591231 11826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:09.592384 11826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "06774a10fd764f1cbfd7fc3d15f2173e"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "ae118348c8c8e0ad5d714d06d5b29bf6"
server_key_iv: "63f79132740b929d55436ba923ced4ff"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.592779 11826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "06774a10fd764f1cbfd7fc3d15f2173e"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "ae118348c8c8e0ad5d714d06d5b29bf6"
server_key_iv: "63f79132740b929d55436ba923ced4ff"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.594385 11826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:15:09.595685 11841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.596163 11826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:15:09.596284 11826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "06774a10fd764f1cbfd7fc3d15f2173e"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "ae118348c8c8e0ad5d714d06d5b29bf6"
server_key_iv: "63f79132740b929d55436ba923ced4ff"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.596379 11826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:09.621809 11826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:09.622186 11826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:09.622350 11826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:09.622606 11826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:09.623001 11826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:09.623059 11826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.623093 11826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:09.623121 11826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.628829 11826 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:37237
I20260619 08:15:09.628875 11955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:37237 every 8 connection(s)
I20260619 08:15:09.629344 11826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:15:09.630640 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11826
I20260619 08:15:09.630738 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:15:09.630976 2527 external_mini_cluster.cc:1428] Setting key 843ba962e2e2ca87775b672cff98b1dc
I20260619 08:15:09.632153 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--builtin_ntp_servers=127.2.119.212:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 08:15:09.636338 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.630168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:40837 (local address 127.2.119.254:37437)
0619 08:15:09.630269 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.630271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.630538 (+ 267us) server_negotiation.cc:407] Connection header received
0619 08:15:09.631297 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.631299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.631313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.631389 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.632221 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.632979 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.634721 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.634911 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.634970 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.636076 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:09.636077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:09.636077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:09.636082 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:09.636096 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:09.636100 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:09.636148 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:09.636251 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:15:09.636278 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 08:15:09.636636 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.630054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37437 (local address 127.2.119.193:40837)
0619 08:15:09.630351 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:09.630400 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:09.631153 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:09.631482 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:09.631486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:09.631504 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:09.632120 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.632125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.633229 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.633230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.634623 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.634626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.635021 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.635022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.635232 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.635915 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:09.635915 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:09.635957 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:09.635988 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:09.636150 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:09.636155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:09.636158 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:15:09.636233 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 08:15:09.636358 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260619 08:15:09.637230 11956 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37437
I20260619 08:15:09.637358 11956 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:09.637578 11956 heartbeater.cc:511] Master 127.2.119.254:37437 requested a full tablet report, sending...
I20260619 08:15:09.638091 11766 ts_manager.cc:194] Registered new tserver with Master: 06774a10fd764f1cbfd7fc3d15f2173e (127.2.119.193:37237)
I20260619 08:15:09.638734 11766 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.193:40837
W20260619 08:15:09.715996 11959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:09.716226 11959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:09.716249 11959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:09.717489 11959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:09.717530 11959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:09.717555 11959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:15:09.718907 11959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:09.719177 11959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:09.719491 11959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:09.724789 11965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.724874 11967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.725023 11964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:09.725175 11959 server_base.cc:1034] running on GCE node
I20260619 08:15:09.725438 11959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:09.725697 11959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:09.726852 11959 hybrid_clock.cc:648] HybridClock initialized: now 1781856909726832 us; error 39 us; skew 500 ppm
I20260619 08:15:09.728274 11959 webserver.cc:460] Webserver started at http://127.2.119.194:40661/ using document root <none> and password file <none>
I20260619 08:15:09.728538 11959 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:09.728601 11959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:09.728742 11959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:09.729965 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "d9154e040f3d41c19e028bf9ccd93a78"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "a113932a062a8ffb421bcc96125bf662"
server_key_iv: "1b05944e0b2fc6eabb199faafe6fd915"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.730360 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "d9154e040f3d41c19e028bf9ccd93a78"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "a113932a062a8ffb421bcc96125bf662"
server_key_iv: "1b05944e0b2fc6eabb199faafe6fd915"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.731900 11959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:15:09.732853 11973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.733044 11959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:09.733139 11959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "d9154e040f3d41c19e028bf9ccd93a78"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "a113932a062a8ffb421bcc96125bf662"
server_key_iv: "1b05944e0b2fc6eabb199faafe6fd915"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.733253 11959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:09.753364 11959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:09.753736 11959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:09.753881 11959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:09.754149 11959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:09.754528 11959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:09.754565 11959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.754587 11959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:09.754601 11959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.760668 11959 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:42287
I20260619 08:15:09.760775 12086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:42287 every 8 connection(s)
I20260619 08:15:09.761098 11959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:15:09.766500 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.761966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:35179 (local address 127.2.119.254:37437)
0619 08:15:09.762067 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.762135 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.762276 (+ 141us) server_negotiation.cc:407] Connection header received
0619 08:15:09.762836 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.762837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.762853 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.762918 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.763603 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.764107 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.765109 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.765322 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.765431 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.766221 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:09.766223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:09.766223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:09.766228 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:09.766243 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:09.766247 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:09.766299 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:09.766390 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:15:09.766420 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 08:15:09.766762 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.761867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37437 (local address 127.2.119.194:35179)
0619 08:15:09.762131 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:09.762178 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:09.762727 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:09.762990 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:09.762994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:09.763004 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:09.763496 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.763500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.764201 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.764202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.765001 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.765003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.765437 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.765439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.765579 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.766087 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:09.766087 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:09.766116 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:09.766138 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:09.766306 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:09.766311 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:09.766314 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:15:09.766392 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 08:15:09.766494 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":126,"threads_started":1}
I20260619 08:15:09.767211 12087 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37437
I20260619 08:15:09.767299 12087 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:09.767441 12087 heartbeater.cc:511] Master 127.2.119.254:37437 requested a full tablet report, sending...
I20260619 08:15:09.767799 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 11959
I20260619 08:15:09.767849 11766 ts_manager.cc:194] Registered new tserver with Master: d9154e040f3d41c19e028bf9ccd93a78 (127.2.119.194:42287)
I20260619 08:15:09.767907 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:15:09.768131 2527 external_mini_cluster.cc:1428] Setting key 8b39b9002c00a5d16831e6bc3871dc48
I20260619 08:15:09.768337 11766 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.194:35179
I20260619 08:15:09.769248 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--builtin_ntp_servers=127.2.119.212:45125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 08:15:09.850366 12090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:09.850615 12090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:09.850648 12090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:09.851805 12090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:09.851876 12090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:09.851917 12090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:15:09.853235 12090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:09.853533 12090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:09.853837 12090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:09.859335 12095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.859289 12098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:09.859289 12096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:09.859414 12090 server_base.cc:1034] running on GCE node
I20260619 08:15:09.859766 12090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:09.860024 12090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:09.861166 12090 hybrid_clock.cc:648] HybridClock initialized: now 1781856909861154 us; error 32 us; skew 500 ppm
I20260619 08:15:09.862511 12090 webserver.cc:460] Webserver started at http://127.2.119.195:37677/ using document root <none> and password file <none>
I20260619 08:15:09.862752 12090 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:09.862802 12090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:09.862933 12090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:09.863986 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "ab54253166394f00a75da410e9f6d7fd"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "22b80bfbd2df996e1c454495fc055581"
server_key_iv: "dc89eb4e99516bd5d30ae50ea9468423"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.864341 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "ab54253166394f00a75da410e9f6d7fd"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "22b80bfbd2df996e1c454495fc055581"
server_key_iv: "dc89eb4e99516bd5d30ae50ea9468423"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.865691 12090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 08:15:09.866444 12104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.866699 12090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:09.866797 12090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "ab54253166394f00a75da410e9f6d7fd"
format_stamp: "Formatted at 2026-06-19 08:15:09 on dist-test-slave-d0b5"
server_key: "22b80bfbd2df996e1c454495fc055581"
server_key_iv: "dc89eb4e99516bd5d30ae50ea9468423"
server_key_version: "encryptionkey@0"
I20260619 08:15:09.866868 12090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:09.881845 12090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:09.882198 12090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:09.882364 12090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:09.882616 12090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:09.882968 12090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:09.883018 12090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.883044 12090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:09.883057 12090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:09.889720 12090 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:42019
I20260619 08:15:09.889761 12217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:42019 every 8 connection(s)
I20260619 08:15:09.890148 12090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:15:09.894033 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12090
I20260619 08:15:09.894109 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:15:09.894371 2527 external_mini_cluster.cc:1428] Setting key 089221d1f8f5b344366f6ebfd62f7fab
I20260619 08:15:09.895926 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.890901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:41385 (local address 127.2.119.254:37437)
0619 08:15:09.890955 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.890958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.891227 (+ 269us) server_negotiation.cc:407] Connection header received
0619 08:15:09.891766 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.891767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.891779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.891852 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.892570 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.893076 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.894356 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.894616 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.894720 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.895681 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:09.895683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:09.895683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:09.895687 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:09.895703 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:09.895708 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:09.895756 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:09.895835 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 08:15:09.895863 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:15:09.896160 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:09.890910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37437 (local address 127.2.119.195:41385)
0619 08:15:09.891094 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:09.891132 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:09.891645 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:09.891907 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:09.891910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:09.891919 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:09.892468 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.892472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.893249 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.893250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.894267 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:09.894270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:09.894723 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:09.894725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:09.894865 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:09.895499 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:09.895500 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:09.895528 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:09.895552 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:09.895771 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:09.895776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:09.895778 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:15:09.895829 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:15:09.895911 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260619 08:15:09.896675 12218 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37437
I20260619 08:15:09.896755 12218 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:09.896968 12218 heartbeater.cc:511] Master 127.2.119.254:37437 requested a full tablet report, sending...
I20260619 08:15:09.897389 11766 ts_manager.cc:194] Registered new tserver with Master: ab54253166394f00a75da410e9f6d7fd (127.2.119.195:42019)
I20260619 08:15:09.897804 11766 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.195:41385
I20260619 08:15:09.906741 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:15:09.908941 11806 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:09.908364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.2.119.254:37437)
0619 08:15:09.908552 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:09.908554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:09.908571 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:15:09.908593 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:09.908594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:09.908605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:09.908661 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:09.908838 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43036: BlockingRecv error: recv got EOF from 127.0.0.1:43036 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
I20260619 08:15:09.909054 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11826
I20260619 08:15:09.913853 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11959
I20260619 08:15:09.918438 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12090
I20260619 08:15:09.922629 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 11735
2026-06-19T08:15:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (621 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T08:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:15:09Z Disabled control of system clock
I20260619 08:15:09.949671 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36683
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:15:10.030247 12229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:10.030458 12229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:10.030479 12229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:10.031551 12229 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:15:10.031594 12229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:10.031607 12229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:10.031620 12229 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:15:10.031631 12229 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:15:10.033146 12229 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:10.033432 12229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:10.033707 12229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:10.039814 12237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.039803 12235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.039748 12229 server_base.cc:1034] running on GCE node
W20260619 08:15:10.039770 12234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.040263 12229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:10.040513 12229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:10.041656 12229 hybrid_clock.cc:648] HybridClock initialized: now 1781856910041636 us; error 35 us; skew 500 ppm
I20260619 08:15:10.042811 12229 webserver.cc:460] Webserver started at http://127.2.119.254:33341/ using document root <none> and password file <none>
I20260619 08:15:10.043073 12229 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:10.043145 12229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:10.043290 12229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:10.044363 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "7554ddf8fcaf4204aa5c8538722279db"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "ec5694dd96c61395732677ad4a0fbc5c"
server_key_iv: "96e95b9576c12f7cbf1b9d8be2b13477"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.044736 12229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "7554ddf8fcaf4204aa5c8538722279db"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "ec5694dd96c61395732677ad4a0fbc5c"
server_key_iv: "96e95b9576c12f7cbf1b9d8be2b13477"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.046151 12229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:15:10.046952 12243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.047173 12229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:15:10.047277 12229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "7554ddf8fcaf4204aa5c8538722279db"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "ec5694dd96c61395732677ad4a0fbc5c"
server_key_iv: "96e95b9576c12f7cbf1b9d8be2b13477"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.047359 12229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:10.050868 12229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:10.051143 12229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:10.051285 12229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:10.055161 12229 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36683
I20260619 08:15:10.055258 12295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36683 every 8 connection(s)
I20260619 08:15:10.055549 12229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:15:10.056257 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:10.059119 12296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db: Bootstrap starting.
I20260619 08:15:10.059722 12296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:10.060021 12296 log.cc:824] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:10.060767 12296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db: No bootstrap required, opened a new log
I20260619 08:15:10.062485 12296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } }
I20260619 08:15:10.062619 12296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:10.062641 12296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7554ddf8fcaf4204aa5c8538722279db, State: Initialized, Role: FOLLOWER
I20260619 08:15:10.062788 12296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } }
I20260619 08:15:10.062860 12296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:15:10.062893 12296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:15:10.062959 12296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:10.063499 12296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } }
I20260619 08:15:10.063583 12296 leader_election.cc:302] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7554ddf8fcaf4204aa5c8538722279db; no voters:
I20260619 08:15:10.063812 12296 leader_election.cc:288] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:15:10.063843 12299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:10.063925 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12229
I20260619 08:15:10.063957 12299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [term 1 LEADER]: Becoming Leader. State: Replica: 7554ddf8fcaf4204aa5c8538722279db, State: Running, Role: LEADER
I20260619 08:15:10.064023 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:15:10.064079 12296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:15:10.064028 12299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } }
I20260619 08:15:10.064240 2527 external_mini_cluster.cc:1428] Setting key c67cbef7bcec39bf590c5d8760259676
I20260619 08:15:10.064471 12301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7554ddf8fcaf4204aa5c8538722279db. Latest consensus state: current_term: 1 leader_uuid: "7554ddf8fcaf4204aa5c8538722279db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } } }
I20260619 08:15:10.064561 12301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:10.064997 12300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7554ddf8fcaf4204aa5c8538722279db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7554ddf8fcaf4204aa5c8538722279db" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36683 } } }
I20260619 08:15:10.065076 12300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:10.065439 12304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:15:10.065943 12304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:15:10.067155 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.064905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.2.119.254:36683)
0619 08:15:10.065536 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:10.065545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:10.065576 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:15:10.065703 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:10.065712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:10.065762 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:10.065894 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:10.066657 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:10.066661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:10.066661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:10.066665 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:10.066711 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:10.066727 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:10.066796 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:10.066870 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:15:10.066930 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":74,"threads_started":1}
I20260619 08:15:10.067973 12304 catalog_manager.cc:1347] Generated new cluster ID: 88bfa13182dc4431bd71681d78a4a144
I20260619 08:15:10.068022 12304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:15:10.090266 12304 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:15:10.090483 12304 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:15:10.100071 12304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7554ddf8fcaf4204aa5c8538722279db: Generated new TSK 0
I20260619 08:15:10.100287 12304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:15:10.108634 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--builtin_ntp_servers=127.2.119.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:15:10.194572 12320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:10.194808 12320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:10.194845 12320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:10.196012 12320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:10.196076 12320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:10.196122 12320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:15:10.197569 12320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:10.197834 12320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:10.198127 12320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:10.203297 12328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.203398 12325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.203526 12326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.203555 12320 server_base.cc:1034] running on GCE node
I20260619 08:15:10.203812 12320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:10.204102 12320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:10.205284 12320 hybrid_clock.cc:648] HybridClock initialized: now 1781856910205262 us; error 39 us; skew 500 ppm
I20260619 08:15:10.206660 12320 webserver.cc:460] Webserver started at http://127.2.119.193:45707/ using document root <none> and password file <none>
I20260619 08:15:10.206935 12320 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:10.207012 12320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:10.207175 12320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:10.208375 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "048e0ad5ee1940c5af207ca8db332ef2"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "fe1e951fa793b274cc8e27aa4dcb4767"
server_key_iv: "d2523da4e5e7c280b3dc6c935df08053"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.208748 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "048e0ad5ee1940c5af207ca8db332ef2"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "fe1e951fa793b274cc8e27aa4dcb4767"
server_key_iv: "d2523da4e5e7c280b3dc6c935df08053"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.210291 12320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 08:15:10.211398 12334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.211656 12320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:10.211762 12320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "048e0ad5ee1940c5af207ca8db332ef2"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "fe1e951fa793b274cc8e27aa4dcb4767"
server_key_iv: "d2523da4e5e7c280b3dc6c935df08053"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.211858 12320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:10.215382 12320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:10.215649 12320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:10.215795 12320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:10.216002 12320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:10.216310 12320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:10.216367 12320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.216405 12320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:10.216434 12320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.222051 12320 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:45773
I20260619 08:15:10.222093 12447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:45773 every 8 connection(s)
I20260619 08:15:10.222401 12320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:15:10.223737 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12320
I20260619 08:15:10.223822 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:15:10.224043 2527 external_mini_cluster.cc:1428] Setting key d434bf358db9985ee6a40d8067e16d4d
I20260619 08:15:10.225049 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--builtin_ntp_servers=127.2.119.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 08:15:10.225674 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.223121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:50321 (local address 127.2.119.254:36683)
0619 08:15:10.223219 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:10.223222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:10.223410 (+ 188us) server_negotiation.cc:407] Connection header received
0619 08:15:10.223913 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:10.223914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:10.223924 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:10.224012 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:10.225391 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:10.225392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:10.225393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:10.225398 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:10.225416 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:10.225421 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:10.225488 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:10.225548 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:15:10.225564 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 08:15:10.225814 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.223096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36683 (local address 127.2.119.193:50321)
0619 08:15:10.223285 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:10.223331 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:10.223760 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:10.224065 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:10.224069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:10.224092 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:10.225168 (+ 1076us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:10.225169 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:10.225214 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:10.225254 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:10.225494 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:10.225497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:10.225499 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:15:10.225553 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:15:10.225601 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260619 08:15:10.226451 12448 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36683
I20260619 08:15:10.226617 12448 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:10.226788 12448 heartbeater.cc:511] Master 127.2.119.254:36683 requested a full tablet report, sending...
I20260619 08:15:10.227605 12260 ts_manager.cc:194] Registered new tserver with Master: 048e0ad5ee1940c5af207ca8db332ef2 (127.2.119.193:45773)
W20260619 08:15:10.306764 12451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:10.306962 12451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:10.306984 12451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:10.308159 12451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:10.308199 12451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:10.308224 12451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:15:10.309556 12451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:10.309808 12451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:10.310117 12451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:10.315173 12457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.315342 12451 server_base.cc:1034] running on GCE node
W20260619 08:15:10.315181 12459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.315166 12456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.315698 12451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:10.315958 12451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:10.317185 12451 hybrid_clock.cc:648] HybridClock initialized: now 1781856910317172 us; error 30 us; skew 500 ppm
I20260619 08:15:10.318583 12451 webserver.cc:460] Webserver started at http://127.2.119.194:41377/ using document root <none> and password file <none>
I20260619 08:15:10.318827 12451 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:10.318897 12451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:10.319036 12451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:10.320142 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "8692241b667246e78d4207a227438015"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e4aefa0200d1b6f889de5d48a11e6042"
server_key_iv: "d966258bdce5103ad04d57b52511ddad"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.320510 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "8692241b667246e78d4207a227438015"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e4aefa0200d1b6f889de5d48a11e6042"
server_key_iv: "d966258bdce5103ad04d57b52511ddad"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.321888 12451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:15:10.322700 12465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.322871 12451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:15:10.322974 12451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "8692241b667246e78d4207a227438015"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e4aefa0200d1b6f889de5d48a11e6042"
server_key_iv: "d966258bdce5103ad04d57b52511ddad"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.323057 12451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:10.326479 12451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:10.326731 12451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:10.326864 12451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:10.327071 12451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:10.327364 12451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:10.327418 12451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.327456 12451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:10.327483 12451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.333326 12451 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:38789
I20260619 08:15:10.333369 12578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:38789 every 8 connection(s)
I20260619 08:15:10.333676 12451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:15:10.336602 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.334458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:56285 (local address 127.2.119.254:36683)
0619 08:15:10.334528 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:10.334530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:10.334727 (+ 197us) server_negotiation.cc:407] Connection header received
0619 08:15:10.335210 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:10.335211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:10.335217 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:10.335278 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:10.336357 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:10.336358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:10.336359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:10.336363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:10.336378 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:10.336382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:10.336434 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:10.336537 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:15:10.336549 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:15:10.336825 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.334432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36683 (local address 127.2.119.194:56285)
0619 08:15:10.334605 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:10.334643 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:10.335101 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:10.335326 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:10.335330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:10.335337 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:10.336210 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:10.336210 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:10.336237 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:10.336264 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:10.336467 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:10.336470 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:10.336472 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:15:10.336530 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:15:10.336577 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260619 08:15:10.337386 12579 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36683
I20260619 08:15:10.337495 12579 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:10.337639 12579 heartbeater.cc:511] Master 127.2.119.254:36683 requested a full tablet report, sending...
I20260619 08:15:10.338030 12260 ts_manager.cc:194] Registered new tserver with Master: 8692241b667246e78d4207a227438015 (127.2.119.194:38789)
I20260619 08:15:10.340404 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12451
I20260619 08:15:10.340488 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:15:10.340692 2527 external_mini_cluster.cc:1428] Setting key ce84d0282afb9cd2a3f477628b344a68
I20260619 08:15:10.341627 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--builtin_ntp_servers=127.2.119.212:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 08:15:10.422434 12582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:10.422631 12582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:10.422654 12582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:10.423772 12582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:10.423804 12582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:10.423830 12582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:15:10.425052 12582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:10.425239 12582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:10.425597 12582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:10.430408 12587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.430449 12588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:10.430428 12590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:10.430666 12582 server_base.cc:1034] running on GCE node
I20260619 08:15:10.430927 12582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:10.431181 12582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:10.432387 12582 hybrid_clock.cc:648] HybridClock initialized: now 1781856910432375 us; error 30 us; skew 500 ppm
I20260619 08:15:10.433904 12582 webserver.cc:460] Webserver started at http://127.2.119.195:43207/ using document root <none> and password file <none>
I20260619 08:15:10.434166 12582 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:10.434242 12582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:10.434406 12582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:10.435613 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "aa8dec1ec0d24413aed83847e48e211b"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e46fb5fcac4da5d6aad41e7cd907a7f2"
server_key_iv: "8aff9c55673ae02d1ec735acb2cec473"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.436053 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "aa8dec1ec0d24413aed83847e48e211b"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e46fb5fcac4da5d6aad41e7cd907a7f2"
server_key_iv: "8aff9c55673ae02d1ec735acb2cec473"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.437458 12582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:15:10.438238 12596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.438423 12582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:10.438519 12582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "aa8dec1ec0d24413aed83847e48e211b"
format_stamp: "Formatted at 2026-06-19 08:15:10 on dist-test-slave-d0b5"
server_key: "e46fb5fcac4da5d6aad41e7cd907a7f2"
server_key_iv: "8aff9c55673ae02d1ec735acb2cec473"
server_key_version: "encryptionkey@0"
I20260619 08:15:10.438609 12582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:10.441995 12582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:10.442273 12582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:10.442410 12582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:10.442613 12582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:10.442907 12582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:10.442960 12582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.443001 12582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:10.443028 12582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:10.448637 12582 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:42335
I20260619 08:15:10.448701 12710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:42335 every 8 connection(s)
I20260619 08:15:10.449036 12582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:15:10.452190 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.449895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:33517 (local address 127.2.119.254:36683)
0619 08:15:10.449979 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:10.449982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:10.450265 (+ 283us) server_negotiation.cc:407] Connection header received
0619 08:15:10.450713 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:10.450714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:10.450735 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:10.450825 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:10.451984 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:10.451985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:10.451985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:15:10.451989 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:10.452002 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:15:10.452006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:10.452061 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:10.452123 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:15:10.452137 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:15:10.452407 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:10.449870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36683 (local address 127.2.119.195:33517)
0619 08:15:10.450050 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:10.450099 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:10.450553 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:10.450877 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:10.450880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:10.450888 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:10.451847 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:15:10.451848 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:10.451875 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:15:10.451903 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:10.452061 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:10.452063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:10.452065 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:15:10.452124 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:15:10.452169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260619 08:15:10.452915 12711 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36683
I20260619 08:15:10.453003 12711 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:10.453112 12711 heartbeater.cc:511] Master 127.2.119.254:36683 requested a full tablet report, sending...
I20260619 08:15:10.453656 12260 ts_manager.cc:194] Registered new tserver with Master: aa8dec1ec0d24413aed83847e48e211b (127.2.119.195:42335)
I20260619 08:15:10.456354 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12582
I20260619 08:15:10.456434 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:15:10.456578 2527 external_mini_cluster.cc:1428] Setting key ce459fd686678ffc80fe3456f32d8dd8
I20260619 08:15:10.457470 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 08:15:10.459244 12308 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:15:10.458756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.2.119.254:36683)
0619 08:15:10.458844 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:10.458847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:10.458920 (+ 73us) server_negotiation.cc:407] Connection header received
0619 08:15:10.458992 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:10.458993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:10.458998 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:10.459055 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:10.459172 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60542: BlockingRecv error: recv got EOF from 127.0.0.1:60542 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:15:10.459486 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12320
I20260619 08:15:10.464818 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12451
I20260619 08:15:10.468936 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12582
I20260619 08:15:10.473059 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12229
2026-06-19T08:15:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (550 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:15:10 dist-test-slave-d0b5 krb5kdc[12722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:15:10 dist-test-slave-d0b5 krb5kdc[12722](info): set up 2 sockets
Jun 19 08:15:10 dist-test-slave-d0b5 krb5kdc[12722](info): commencing operation
krb5kdc: starting...
W20260619 08:15:11.531278 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:15:11 dist-test-slave-d0b5 krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:15:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:15:11.651815 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:44781
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:44781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:11.730384 12735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:11.730584 12735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:11.730607 12735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:11.731714 12735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:15:11.731750 12735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:11.731765 12735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:11.731776 12735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:15:11.731788 12735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:15:11.733172 12735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:44781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:44781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:11.733426 12735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:11.733610 12735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:11.738620 12743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:11.738638 12740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:11.738634 12741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:11.738713 12735 server_base.cc:1034] running on GCE node
I20260619 08:15:11.739108 12735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:11.739394 12735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:11.740557 12735 hybrid_clock.cc:648] HybridClock initialized: now 1781856911740535 us; error 42 us; skew 500 ppm
Jun 19 08:15:11 dist-test-slave-d0b5 krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856911, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:11.742995 12735 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:15:11.743465 12735 webserver.cc:460] Webserver started at http://127.2.119.254:43469/ using document root <none> and password file <none>
I20260619 08:15:11.743709 12735 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:11.743778 12735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:11.743927 12735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:11.744961 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "be650a45780d4069a0e401d960ecc494"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "d2c0b5def20f71af57f3ccd7d7932eda"
server_key_iv: "28d61bc9731b9378b56132ff4a977d31"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.745369 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "be650a45780d4069a0e401d960ecc494"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "d2c0b5def20f71af57f3ccd7d7932eda"
server_key_iv: "28d61bc9731b9378b56132ff4a977d31"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.746686 12735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:11.747416 12750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:11.747609 12735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:15:11.747714 12735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "be650a45780d4069a0e401d960ecc494"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "d2c0b5def20f71af57f3ccd7d7932eda"
server_key_iv: "28d61bc9731b9378b56132ff4a977d31"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.747792 12735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:11.760448 12735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:11.762506 12735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:11.762681 12735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:11.766634 12735 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:44781
I20260619 08:15:11.766669 12802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:44781 every 8 connection(s)
I20260619 08:15:11.767012 12735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:15:11.767938 12803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:11.770226 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: Bootstrap starting.
I20260619 08:15:11.770817 12803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:11.771114 12803 log.cc:824] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:11.771826 12803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: No bootstrap required, opened a new log
I20260619 08:15:11.773509 12803 raft_consensus.cc:348] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } }
I20260619 08:15:11.773643 12803 raft_consensus.cc:374] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:11.773664 12803 raft_consensus.cc:729] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be650a45780d4069a0e401d960ecc494, State: Initialized, Role: FOLLOWER
I20260619 08:15:11.773828 12803 consensus_queue.cc:260] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } }
I20260619 08:15:11.773900 12803 raft_consensus.cc:388] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:15:11.773942 12803 raft_consensus.cc:482] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:15:11.773993 12803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:11.774770 12803 raft_consensus.cc:504] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } }
I20260619 08:15:11.774854 12803 leader_election.cc:302] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be650a45780d4069a0e401d960ecc494; no voters:
I20260619 08:15:11.775043 12803 leader_election.cc:288] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:15:11.775085 12806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:11.775295 12806 raft_consensus.cc:686] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [term 1 LEADER]: Becoming Leader. State: Replica: be650a45780d4069a0e401d960ecc494, State: Running, Role: LEADER
I20260619 08:15:11.775332 12803 sys_catalog.cc:564] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:15:11.775388 12806 consensus_queue.cc:237] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } }
I20260619 08:15:11.775820 12808 sys_catalog.cc:455] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be650a45780d4069a0e401d960ecc494. Latest consensus state: current_term: 1 leader_uuid: "be650a45780d4069a0e401d960ecc494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } } }
I20260619 08:15:11.775920 12808 sys_catalog.cc:458] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:11.776103 12815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:15:11.775801 12807 sys_catalog.cc:455] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be650a45780d4069a0e401d960ecc494" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be650a45780d4069a0e401d960ecc494" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 44781 } } }
I20260619 08:15:11.776363 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12735
I20260619 08:15:11.776471 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:15:11.776497 12807 sys_catalog.cc:458] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:11.776707 2527 external_mini_cluster.cc:1428] Setting key f8ea9ff4d8255b857dd9e6fdfdb904f0
I20260619 08:15:11.777184 12815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:15:11.778856 12815 catalog_manager.cc:1347] Generated new cluster ID: d682d52fc53846229fb1d5ad84dc0061
I20260619 08:15:11.778910 12815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:15:11 dist-test-slave-d0b5 krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:11.787978 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:11.777199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.2.119.254:44781)
0619 08:15:11.777582 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:11.777591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:11.777620 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:15:11.777806 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:11.777813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:11.777866 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:11.777995 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:11.778753 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.780139 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.781157 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.781754 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.781823 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:11.784488 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:11.784492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:11.784492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:11.784502 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:11.786392 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:11.786755 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:11.786758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:11.786759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:11.786848 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:11.787014 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:11.787015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:11.787015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:11.787292 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:11.787392 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:11.787611 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 08:15:11.787745 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":95,"threads_started":1}
I20260619 08:15:11.809340 12815 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:15:11.809966 12815 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:15:11.819309 12815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: Generated new TSK 0
I20260619 08:15:11.819530 12815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:15:11.853940 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:11.944003 12829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:11.944209 12829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:11.944233 12829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:11.945412 12829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:11.945454 12829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:11.945480 12829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:15:11.946910 12829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:11.947086 12829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:11.947260 12829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:11.952450 12834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:11.952466 12835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:11.952651 12829 server_base.cc:1034] running on GCE node
W20260619 08:15:11.952452 12837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:11.952934 12829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:11.953181 12829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:11.954348 12829 hybrid_clock.cc:648] HybridClock initialized: now 1781856911954332 us; error 27 us; skew 500 ppm
Jun 19 08:15:11 dist-test-slave-d0b5 krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856911, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:11.956895 12829 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:15:11.957396 12829 webserver.cc:460] Webserver started at http://127.2.119.193:33573/ using document root <none> and password file <none>
I20260619 08:15:11.957620 12829 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:11.957688 12829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:11.957851 12829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:11.958952 12829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "833741d059a64a229ef8e504cecd7c2b"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "5da5f4886293152eb9076fb4143252c2"
server_key_iv: "58aa0e322dfea2fe8f7f252030bcd50a"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.959318 12829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "833741d059a64a229ef8e504cecd7c2b"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "5da5f4886293152eb9076fb4143252c2"
server_key_iv: "58aa0e322dfea2fe8f7f252030bcd50a"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.960704 12829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:15:11.961596 12844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:11.961777 12829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:15:11.961876 12829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "833741d059a64a229ef8e504cecd7c2b"
format_stamp: "Formatted at 2026-06-19 08:15:11 on dist-test-slave-d0b5"
server_key: "5da5f4886293152eb9076fb4143252c2"
server_key_iv: "58aa0e322dfea2fe8f7f252030bcd50a"
server_key_version: "encryptionkey@0"
I20260619 08:15:11.961959 12829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:11.983434 12829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:11.985606 12829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:11.985770 12829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:11.986018 12829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:11.986330 12829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:11.986384 12829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:11.986425 12829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:11.986467 12829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:11.992321 12829 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:44325
I20260619 08:15:11.992409 12957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:44325 every 8 connection(s)
I20260619 08:15:11.992741 12829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:15:11 dist-test-slave-d0b5 krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856911, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:11.999464 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12829
I20260619 08:15:11.999571 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:15:11.999782 2527 external_mini_cluster.cc:1428] Setting key 778fdea248b93f04932d459e3e1878e8
I20260619 08:15:12.002424 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:11.993608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44781 (local address 127.2.119.193:39711)
0619 08:15:11.993771 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:11.993806 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:11.994263 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:11.994482 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:11.994485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:11.995055 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:11.995248 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:11.995252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.996021 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.996021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:11.996798 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:11.996799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.997169 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.997169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:11.997265 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:11.997787 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:11.997795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:11.999174 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:12.000902 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.000909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.000911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.001242 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.001481 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.001482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.001482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.001623 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.001941 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:12.001948 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:12.002061 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.002146 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.002268 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260619 08:15:12.002462 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:11.993608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:39711 (local address 127.2.119.254:44781)
0619 08:15:11.993652 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:11.993654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:11.993913 (+ 259us) server_negotiation.cc:407] Connection header received
0619 08:15:11.994351 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:11.994353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:11.994368 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:11.994432 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:11.995342 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.995936 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.996879 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:11.997091 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:11.997150 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:11.999280 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:11.999281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:11.999281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:11.999287 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:12.000816 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.001354 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.001355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.001356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.001414 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.001703 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.001706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.001707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.001858 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:12.001935 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.002192 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.002339 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:15:12.002940 12958 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44781
I20260619 08:15:12.003036 12958 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:12.003212 12958 heartbeater.cc:511] Master 127.2.119.254:44781 requested a full tablet report, sending...
I20260619 08:15:12.003829 12767 ts_manager.cc:194] Registered new tserver with Master: 833741d059a64a229ef8e504cecd7c2b (127.2.119.193:44325)
I20260619 08:15:12.004886 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:39711
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:15:12.027685 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:12.109148 12963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:12.109362 12963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:12.109386 12963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:12.110441 12963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:12.110476 12963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:12.110502 12963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:15:12.111876 12963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:12.112058 12963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:12.112260 12963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:12.117187 12971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:12.117399 12968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:12.117312 12969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:12.117420 12963 server_base.cc:1034] running on GCE node
I20260619 08:15:12.117740 12963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:12.117980 12963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:12.119159 12963 hybrid_clock.cc:648] HybridClock initialized: now 1781856912119152 us; error 33 us; skew 500 ppm
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856912, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:12.121587 12963 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:15:12.122081 12963 webserver.cc:460] Webserver started at http://127.2.119.194:40709/ using document root <none> and password file <none>
I20260619 08:15:12.122311 12963 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:12.122377 12963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:12.122509 12963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:12.123517 12963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "ecec4c1839994c628be0ea9b1ef644a5"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "d3b1f092d417bb7f1006ef261fbddc86"
server_key_iv: "64f472da7bd9ab81c608f8209ea7595b"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.123879 12963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "ecec4c1839994c628be0ea9b1ef644a5"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "d3b1f092d417bb7f1006ef261fbddc86"
server_key_iv: "64f472da7bd9ab81c608f8209ea7595b"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.125149 12963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:15:12.125973 12978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.126179 12963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:12.126281 12963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "ecec4c1839994c628be0ea9b1ef644a5"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "d3b1f092d417bb7f1006ef261fbddc86"
server_key_iv: "64f472da7bd9ab81c608f8209ea7595b"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.126366 12963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:12.138682 12963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:12.140587 12963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:12.140746 12963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:12.140959 12963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:12.141249 12963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:12.141305 12963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.141422 12963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:12.141453 12963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.147343 12963 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:41493
I20260619 08:15:12.147433 13091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:41493 every 8 connection(s)
I20260619 08:15:12.147712 12963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:15:12.152511 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 12963
I20260619 08:15:12.152621 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:15:12.152830 2527 external_mini_cluster.cc:1428] Setting key f99bdab8fe3d91553a2cc50c3597f6ac
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856912, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:12.157112 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.148426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:46219 (local address 127.2.119.254:44781)
0619 08:15:12.148513 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.148515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.148698 (+ 183us) server_negotiation.cc:407] Connection header received
0619 08:15:12.149197 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.149198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.149208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.149263 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:12.150225 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.150752 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.151783 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.151972 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.152034 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.154316 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:12.154317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:12.154318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:12.154326 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:12.155785 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.156158 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.156159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.156159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.156209 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.156476 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.156477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.156477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.156596 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:12.156673 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.156941 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.157020 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 08:15:12.157356 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.148378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44781 (local address 127.2.119.194:46219)
0619 08:15:12.148556 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.148602 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.149081 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.149364 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.149368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.149957 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:12.150141 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.150145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.150831 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.150831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.151686 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.151688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.152045 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.152046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.152146 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.152600 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:12.152611 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:12.154217 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:12.155865 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.155870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.155872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.156092 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.156286 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.156287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.156287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.156402 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.156664 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:12.156670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:12.156808 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.156920 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.157074 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260619 08:15:12.157866 13092 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44781
I20260619 08:15:12.157974 13092 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:12.158141 13092 heartbeater.cc:511] Master 127.2.119.254:44781 requested a full tablet report, sending...
I20260619 08:15:12.158521 12767 ts_manager.cc:194] Registered new tserver with Master: ecec4c1839994c628be0ea9b1ef644a5 (127.2.119.194:41493)
I20260619 08:15:12.159091 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:46219
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:15:12.180840 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:12.265390 13097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:12.265667 13097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:12.265697 13097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:12.266872 13097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:12.266916 13097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:12.266945 13097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:15:12.268424 13097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:44781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:12.268676 13097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:12.268916 13097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:12.274304 13103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:12.274312 13105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:12.274323 13102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:12.274469 13097 server_base.cc:1034] running on GCE node
I20260619 08:15:12.274748 13097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:12.275048 13097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:12.276237 13097 hybrid_clock.cc:648] HybridClock initialized: now 1781856912276247 us; error 56 us; skew 500 ppm
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[12722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856912, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:12.278823 13097 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:15:12.279369 13097 webserver.cc:460] Webserver started at http://127.2.119.195:33959/ using document root <none> and password file <none>
I20260619 08:15:12.279618 13097 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:12.279685 13097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:12.279834 13097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:15:12.280874 13097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "b3aca3c50b8648c29e6621e8776bf9b3"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "df4f196ca637e584c00e5689c1710fd9"
server_key_iv: "64c90a7d385f4a3f23ea296898544330"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.281234 13097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "b3aca3c50b8648c29e6621e8776bf9b3"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "df4f196ca637e584c00e5689c1710fd9"
server_key_iv: "64c90a7d385f4a3f23ea296898544330"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.282541 13097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:15:12.283310 13112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.283488 13097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:15:12.283589 13097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "b3aca3c50b8648c29e6621e8776bf9b3"
format_stamp: "Formatted at 2026-06-19 08:15:12 on dist-test-slave-d0b5"
server_key: "df4f196ca637e584c00e5689c1710fd9"
server_key_iv: "64c90a7d385f4a3f23ea296898544330"
server_key_version: "encryptionkey@0"
I20260619 08:15:12.283679 13097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:12.298713 13097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:12.301002 13097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:12.301167 13097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:12.301445 13097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:12.301753 13097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:12.301808 13097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.301848 13097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:12.301877 13097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:12.307705 13097 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:37867
I20260619 08:15:12.307762 13226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:37867 every 8 connection(s)
I20260619 08:15:12.308113 13097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[12722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856912, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:12.315685 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 13097
I20260619 08:15:12.315826 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:15:12.316023 2527 external_mini_cluster.cc:1428] Setting key f56533468c1dcfaeea247ca3eb5b25f3
I20260619 08:15:12.317414 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.308832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:58413 (local address 127.2.119.254:44781)
0619 08:15:12.308894 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.308897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.309067 (+ 170us) server_negotiation.cc:407] Connection header received
0619 08:15:12.309592 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.309593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.309602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.309662 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:12.310529 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.311028 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.312031 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.312212 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.312298 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.314509 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:12.314511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:12.314511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:12.314517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:12.315840 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.316294 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.316296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.316296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.316397 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.316681 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.316682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.316682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.316825 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:12.316889 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.317204 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.317268 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:15:12.317569 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.308790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:44781 (local address 127.2.119.195:58413)
0619 08:15:12.308939 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.308979 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.309507 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.309715 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.309718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.310264 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:12.310452 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.310456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.311164 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.311165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.311956 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.311958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.312323 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.312324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.312433 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.312971 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:12.312979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:12.314413 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:12.315920 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.315926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.315928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.316199 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.316476 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:12.316477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:12.316477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:12.316610 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:12.316903 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:12.316908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:12.317068 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.317163 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.317368 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260619 08:15:12.317965 13227 heartbeater.cc:347] Connected to a master server at 127.2.119.254:44781
I20260619 08:15:12.318038 13227 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:12.318166 13227 heartbeater.cc:511] Master 127.2.119.254:44781 requested a full tablet report, sending...
I20260619 08:15:12.318498 12767 ts_manager.cc:194] Registered new tserver with Master: b3aca3c50b8648c29e6621e8776bf9b3 (127.2.119.195:37867)
I20260619 08:15:12.318866 12767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:58413
I20260619 08:15:12.327833 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:15:12.334900 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.329155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.2.119.254:44781)
0619 08:15:12.329218 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.329220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.329445 (+ 225us) server_negotiation.cc:407] Connection header received
0619 08:15:12.329492 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.329493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.329502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.329548 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:12.330175 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.330674 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.331366 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.331543 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.331598 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.332752 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:12.332753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:12.332753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:12.332758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:12.333953 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.334277 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.334278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.334278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.334325 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:12.334481 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:12.334481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:12.334481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:12.334606 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:12.334677 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.334764 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.334817 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 08:15:12.336545 12767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:15:12.338286 12767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:15:12.344537 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.340587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41493 (local address 127.0.0.1:33554)
0619 08:15:12.340872 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.340882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.340921 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.341262 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.341265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.341267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:12.341359 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.341361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342256 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.342257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.343505 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.343508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.344246 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.344247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.344327 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.344341 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.344408 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.344462 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":73,"threads_started":1}
I20260619 08:15:12.344542 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.340701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.2.119.194:41493)
0619 08:15:12.341028 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.341033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.341043 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:15:12.341082 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.341084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.341101 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.341172 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:12.341440 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342153 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.343582 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.344136 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.344250 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.344300 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.344420 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.344457 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":84,"threads_started":1}
I20260619 08:15:12.344679 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.340780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37867 (local address 127.0.0.1:52142)
0619 08:15:12.340910 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.340921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.340974 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.341385 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.341386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.341387 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:12.341421 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.341422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342715 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.342716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.343683 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.343685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.344426 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.344427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.344503 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.344508 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.344565 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.344613 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
I20260619 08:15:12.344947 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.341004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52142 (local address 127.2.119.195:37867)
0619 08:15:12.341182 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.341186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.341195 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:15:12.341229 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.341231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.341254 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.341491 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:12.341552 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342635 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.343775 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.344342 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.344417 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.344469 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.344801 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.344853 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260619 08:15:12.345382 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c (DEFAULT_TABLE table=test-table [id=631e1ba2cd4d4402ba2cc659723a70d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:12.345585 13160 tablet_service.cc:1467] Processing CreateTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c (DEFAULT_TABLE table=test-table [id=631e1ba2cd4d4402ba2cc659723a70d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:12.345770 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93aabbec186c42b58e1fb7bfb441a49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:12.345911 13160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93aabbec186c42b58e1fb7bfb441a49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:12.346264 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.340572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44325 (local address 127.0.0.1:40352)
0619 08:15:12.340727 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.340771 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.340837 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.341297 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.341298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.341300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:12.341381 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.341383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342681 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.342682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.343696 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.343698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.346003 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.346004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.346074 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.346078 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.346146 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.346187 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":54,"threads_started":1}
I20260619 08:15:12.346288 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.340659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40352 (local address 127.2.119.193:44325)
0619 08:15:12.341041 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.341046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.341055 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:15:12.341112 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.341114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.341132 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.341212 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:12.341462 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.342559 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.344506 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.345912 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.345992 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.346048 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.346160 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.346201 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":76,"threads_started":1}
I20260619 08:15:12.346845 12892 tablet_service.cc:1467] Processing CreateTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c (DEFAULT_TABLE table=test-table [id=631e1ba2cd4d4402ba2cc659723a70d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:12.347162 12892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93aabbec186c42b58e1fb7bfb441a49c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:12.348986 13244 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Bootstrap starting.
I20260619 08:15:12.349884 13244 tablet_bootstrap.cc:654] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:12.350234 13246 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Bootstrap starting.
I20260619 08:15:12.350401 13245 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Bootstrap starting.
I20260619 08:15:12.350574 13244 log.cc:824] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:12.350896 13246 tablet_bootstrap.cc:654] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:12.350991 13245 tablet_bootstrap.cc:654] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:12.351449 13244 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: No bootstrap required, opened a new log
I20260619 08:15:12.351493 13246 log.cc:824] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:12.351560 13244 ts_tablet_manager.cc:1397] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260619 08:15:12.351604 13245 log.cc:824] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:12.352290 13246 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: No bootstrap required, opened a new log
I20260619 08:15:12.352295 13245 tablet_bootstrap.cc:492] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: No bootstrap required, opened a new log
I20260619 08:15:12.352358 13245 ts_tablet_manager.cc:1397] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:15:12.352358 13246 ts_tablet_manager.cc:1397] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:15:12.353245 13244 raft_consensus.cc:348] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.353479 13244 raft_consensus.cc:374] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:12.353529 13244 raft_consensus.cc:729] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecec4c1839994c628be0ea9b1ef644a5, State: Initialized, Role: FOLLOWER
I20260619 08:15:12.353652 13244 consensus_queue.cc:260] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.354022 13244 ts_tablet_manager.cc:1428] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 08:15:12.354188 13092 heartbeater.cc:503] Master 127.2.119.254:44781 was elected leader, sending a full tablet report...
I20260619 08:15:12.354287 13245 raft_consensus.cc:348] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.354287 13246 raft_consensus.cc:348] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.354442 13246 raft_consensus.cc:374] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:12.354441 13245 raft_consensus.cc:374] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:12.354485 13245 raft_consensus.cc:729] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3aca3c50b8648c29e6621e8776bf9b3, State: Initialized, Role: FOLLOWER
I20260619 08:15:12.354485 13246 raft_consensus.cc:729] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 833741d059a64a229ef8e504cecd7c2b, State: Initialized, Role: FOLLOWER
I20260619 08:15:12.354588 13245 consensus_queue.cc:260] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.354602 13246 consensus_queue.cc:260] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.354858 13246 ts_tablet_manager.cc:1428] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:15:12.354859 13245 ts_tablet_manager.cc:1428] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:15:12.355003 12958 heartbeater.cc:503] Master 127.2.119.254:44781 was elected leader, sending a full tablet report...
I20260619 08:15:12.355352 13227 heartbeater.cc:503] Master 127.2.119.254:44781 was elected leader, sending a full tablet report...
W20260619 08:15:12.398264 13093 tablet.cc:2369] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:15:12.493484 12959 tablet.cc:2369] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:15:12.558706 13228 tablet.cc:2369] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:15:12.648500 13250 raft_consensus.cc:482] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:15:12.648725 13250 raft_consensus.cc:504] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.649151 13250 leader_election.cc:288] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 833741d059a64a229ef8e504cecd7c2b (127.2.119.193:44325), b3aca3c50b8648c29e6621e8776bf9b3 (127.2.119.195:37867)
I20260619 08:15:12.652208 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.649462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44325 (local address 127.2.119.194:48701)
0619 08:15:12.649532 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.649545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.649593 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.649763 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.649764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.649767 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:12.649843 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.649845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.650513 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.650514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.651398 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.651400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.651935 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.651936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.652023 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.652027 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.652097 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.652134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 08:15:12.652208 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.649547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:48701 (local address 127.2.119.193:44325)
0619 08:15:12.649621 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.649624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.649637 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:15:12.649669 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.649670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.649680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.649747 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:12.649910 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.650432 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.651490 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.651851 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.651939 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.651947 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.652106 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.652141 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:15:12.652544 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.649708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:47785 (local address 127.2.119.195:37867)
0619 08:15:12.649770 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.649772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.649783 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:15:12.649857 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.649859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.649869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.649929 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:12.650106 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.650637 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.651708 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.652129 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.652231 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.652239 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.652432 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.652479 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:15:12.652637 12912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93aabbec186c42b58e1fb7bfb441a49c" candidate_uuid: "ecec4c1839994c628be0ea9b1ef644a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "833741d059a64a229ef8e504cecd7c2b" is_pre_election: true
I20260619 08:15:12.652781 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.649617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37867 (local address 127.2.119.194:47785)
0619 08:15:12.649733 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.649744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:12.649791 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:12.649940 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:12.649941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:12.649942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:12.650020 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.650022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.650733 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.650734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.651599 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:12.651601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.652230 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.652231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:12.652325 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.652331 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:15:12.652654 (+ 323us) client_negotiation.cc:240] Negotiation successful
0619 08:15:12.652702 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260619 08:15:12.652805 12912 raft_consensus.cc:2454] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecec4c1839994c628be0ea9b1ef644a5 in term 0.
I20260619 08:15:12.653061 12981 leader_election.cc:302] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 833741d059a64a229ef8e504cecd7c2b, ecec4c1839994c628be0ea9b1ef644a5; no voters:
I20260619 08:15:12.653208 13250 raft_consensus.cc:2781] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:15:12.653275 13250 raft_consensus.cc:482] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:15:12.653329 13250 raft_consensus.cc:3037] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:12.653445 13180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93aabbec186c42b58e1fb7bfb441a49c" candidate_uuid: "ecec4c1839994c628be0ea9b1ef644a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" is_pre_election: true
I20260619 08:15:12.653549 13180 raft_consensus.cc:2454] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecec4c1839994c628be0ea9b1ef644a5 in term 0.
I20260619 08:15:12.654139 13250 raft_consensus.cc:504] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.654268 13250 leader_election.cc:288] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [CANDIDATE]: Term 1 election: Requested vote from peers 833741d059a64a229ef8e504cecd7c2b (127.2.119.193:44325), b3aca3c50b8648c29e6621e8776bf9b3 (127.2.119.195:37867)
I20260619 08:15:12.654485 13180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93aabbec186c42b58e1fb7bfb441a49c" candidate_uuid: "ecec4c1839994c628be0ea9b1ef644a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3aca3c50b8648c29e6621e8776bf9b3"
I20260619 08:15:12.654551 13180 raft_consensus.cc:3037] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:12.654497 12912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93aabbec186c42b58e1fb7bfb441a49c" candidate_uuid: "ecec4c1839994c628be0ea9b1ef644a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "833741d059a64a229ef8e504cecd7c2b"
I20260619 08:15:12.654608 12912 raft_consensus.cc:3037] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:12.655313 13180 raft_consensus.cc:2454] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecec4c1839994c628be0ea9b1ef644a5 in term 1.
I20260619 08:15:12.655405 12912 raft_consensus.cc:2454] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecec4c1839994c628be0ea9b1ef644a5 in term 1.
I20260619 08:15:12.655469 12981 leader_election.cc:302] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3aca3c50b8648c29e6621e8776bf9b3, ecec4c1839994c628be0ea9b1ef644a5; no voters:
I20260619 08:15:12.655571 13250 raft_consensus.cc:2781] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:12.655736 13250 raft_consensus.cc:686] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 1 LEADER]: Becoming Leader. State: Replica: ecec4c1839994c628be0ea9b1ef644a5, State: Running, Role: LEADER
I20260619 08:15:12.655840 13250 consensus_queue.cc:237] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } }
I20260619 08:15:12.656752 12766 catalog_manager.cc:5495] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecec4c1839994c628be0ea9b1ef644a5 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "ecec4c1839994c628be0ea9b1ef644a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecec4c1839994c628be0ea9b1ef644a5" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41493 } health_report { overall_health: HEALTHY } } }
I20260619 08:15:12.697566 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.695059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.2.119.194:41493)
0619 08:15:12.695186 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:12.695188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:12.695196 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:15:12.695236 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:12.695237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:12.695253 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:12.695334 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:15:12.695482 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.695955 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.696721 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:12.696917 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:12.696992 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:12.697194 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:15:12.697288 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:15:12.697403 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:12.697422 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:15:12.697446 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:15:12.699396 13180 raft_consensus.cc:1261] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 1 FOLLOWER]: Refusing update from remote peer ecec4c1839994c628be0ea9b1ef644a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:15:12.699419 12912 raft_consensus.cc:1261] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 1 FOLLOWER]: Refusing update from remote peer ecec4c1839994c628be0ea9b1ef644a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:15:12.699743 13250 consensus_queue.cc:1035] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "833741d059a64a229ef8e504cecd7c2b" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:15:12.699985 13254 consensus_queue.cc:1035] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3aca3c50b8648c29e6621e8776bf9b3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:15:12.702490 13258 mvcc.cc:205] Tried to move back new op lower bound from 7298485914414096384 to 7298485914239029248. Current Snapshot: MvccSnapshot[applied={T|T < 7298485914414096384}]
I20260619 08:15:12.702483 13259 mvcc.cc:205] Tried to move back new op lower bound from 7298485914414096384 to 7298485914239029248. Current Snapshot: MvccSnapshot[applied={T|T < 7298485914414096384}]
I20260619 08:15:12.703222 13257 mvcc.cc:205] Tried to move back new op lower bound from 7298485914414096384 to 7298485914239029248. Current Snapshot: MvccSnapshot[applied={T|T < 7298485914414096384 or (T in {7298485914414096384})}]
I20260619 08:15:12.704663 12766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:15:12.704762 12766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:15:12.705468 12766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be650a45780d4069a0e401d960ecc494: Sending DeleteTablet for 3 replicas of tablet 93aabbec186c42b58e1fb7bfb441a49c
I20260619 08:15:12.705777 12892 tablet_service.cc:1514] Processing DeleteTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:40352
I20260619 08:15:12.705832 13160 tablet_service.cc:1514] Processing DeleteTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:52142
I20260619 08:15:12.705883 13026 tablet_service.cc:1514] Processing DeleteTablet for tablet 93aabbec186c42b58e1fb7bfb441a49c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:33554
I20260619 08:15:12.706048 13269 tablet_replica.cc:330] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: stopping tablet replica
I20260619 08:15:12.706130 13267 tablet_replica.cc:330] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b: stopping tablet replica
I20260619 08:15:12.706146 13269 raft_consensus.cc:2229] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:15:12.706207 13267 raft_consensus.cc:2229] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:15:12.706231 13269 raft_consensus.cc:2258] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:15:12.706342 13267 raft_consensus.cc:2258] T 93aabbec186c42b58e1fb7bfb441a49c P 833741d059a64a229ef8e504cecd7c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:15:12.706473 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12829
I20260619 08:15:12.706698 13269 ts_tablet_manager.cc:1905] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:15:12.708097 13268 tablet_replica.cc:330] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: stopping tablet replica
I20260619 08:15:12.708168 13268 raft_consensus.cc:2229] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:15:12.708231 13268 raft_consensus.cc:2258] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:15:12.708433 13269 ts_tablet_manager.cc:1918] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:15:12.708492 13269 log.cc:1196] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/93aabbec186c42b58e1fb7bfb441a49c
I20260619 08:15:12.708673 13269 ts_tablet_manager.cc:1939] T 93aabbec186c42b58e1fb7bfb441a49c P ecec4c1839994c628be0ea9b1ef644a5: Deleting consensus metadata
I20260619 08:15:12.709010 12752 catalog_manager.cc:4841] TS ecec4c1839994c628be0ea9b1ef644a5 (127.2.119.194:41493): tablet 93aabbec186c42b58e1fb7bfb441a49c (table test-table [id=631e1ba2cd4d4402ba2cc659723a70d7]) successfully deleted
I20260619 08:15:12.709221 13268 ts_tablet_manager.cc:1905] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:15:12.710613 13268 ts_tablet_manager.cc:1918] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:15:12.710666 13268 log.cc:1196] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/93aabbec186c42b58e1fb7bfb441a49c
I20260619 08:15:12.710891 13268 ts_tablet_manager.cc:1939] T 93aabbec186c42b58e1fb7bfb441a49c P b3aca3c50b8648c29e6621e8776bf9b3: Deleting consensus metadata
I20260619 08:15:12.711293 12753 catalog_manager.cc:4841] TS b3aca3c50b8648c29e6621e8776bf9b3 (127.2.119.195:37867): tablet 93aabbec186c42b58e1fb7bfb441a49c (table test-table [id=631e1ba2cd4d4402ba2cc659723a70d7]) successfully deleted
W20260619 08:15:12.716955 12753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:44325 (error 108)
I20260619 08:15:12.717252 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12963
I20260619 08:15:12.718400 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:12.717568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44325 (local address 127.0.0.1:40368)
0619 08:15:12.718307 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:12.718355 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715}
W20260619 08:15:12.718570 12753 catalog_manager.cc:4584] TS 833741d059a64a229ef8e504cecd7c2b (127.2.119.193:44325): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93aabbec186c42b58e1fb7bfb441a49c: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44325: connect: Connection refused (error 111)
I20260619 08:15:12.722767 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 13097
I20260619 08:15:12.727360 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 12735
2026-06-19T08:15:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2255 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[13273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[13273](info): set up 2 sockets
Jun 19 08:15:12 dist-test-slave-d0b5 krb5kdc[13273](info): commencing operation
krb5kdc: starting...
W20260619 08:15:13.771407 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:15:13 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:15:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangeradmin_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangeradmin_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangerlookup_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangerlookup_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
I20260619 08:15:13.939726 2527 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmYJIvA/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/postgres -l logfile start
2026-06-19 08:15:16.619 UTC [13304] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:15:16.619 UTC [13304] LOG: listening on IPv4 address "127.2.119.212", port 58419
2026-06-19 08:15:16.623 UTC [13304] LOG: listening on Unix socket "/tmp/.s.PGSQL.58419"
2026-06-19 08:15:16.659 UTC [13307] LOG: database system was shut down at 2026-06-19 08:15:15 UTC
2026-06-19 08:15:16.663 UTC [13304] LOG: database system is ready to accept connections
I20260619 08:15:17.631322 2527 mini_postgres.cc:96] Postgres bound to 58419
2026-06-19 08:15:17.638 UTC [13315] FATAL: database "slave" does not exist
127.2.119.212:58419 - accepting connections
I20260619 08:15:17.639626 2527 mini_ranger.cc:161] Starting Ranger...
I20260619 08:15:17.654769 2527 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:15:18.175845 2527 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:15:18.175937 2527 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin
2026-06-19 08:15:18,456 [I] DB FLAVOR :POSTGRES
2026-06-19 08:15:18,457 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:15:18,457 [I] Checking connection..
2026-06-19 08:15:18,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:15:18,708 [I] Checking connection passed.
2026-06-19 08:15:18,708 [I] --------- Verifying version history table ---------
2026-06-19 08:15:18,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:15:18,956 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:15:18,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:15:19,203 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:15:19,203 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:15:19,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:15:19.434 UTC [13379] WARNING: there is no transaction in progress
2026-06-19 08:15:19,440 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:15:19,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:15:19,677 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:15:19,677 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:15:19,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:15:19,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:15:20,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-d0b5.c.gcp-upstream.internal','N') ;"
2026-06-19 08:15:20,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:15:20,656 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:15:20,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:15:20,907 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:15:20,907 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:15:21,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:15:21,409 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:15:21,409 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:15:22.449 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.468 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.486 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.502 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.519 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.540 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.602 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.610 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.626 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.637 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.649 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.664 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.675 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.682 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.691 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:22.699 UTC [13487] WARNING: there is no transaction in progress
2026-06-19 08:15:23,391 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:15:23,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:15:23,640 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:15:23,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:15:23,875 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:15:23,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:15:24,112 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:15:24,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:15:24,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:15:24,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-d0b5.c.gcp-upstream.internal';"
2026-06-19 08:15:24,851 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:15:24,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:15:25,103 [I] DB_PATCHES have already been applied
I20260619 08:15:25.108839 2527 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmYJIvA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:15:25.108940 2527 mini_ranger.cc:193] Using host: 127.2.119.212
I20260619 08:15:25.112077 2527 mini_ranger.cc:227] Ranger admin URL: http://127.2.119.212:32881
Jun 19, 2026 8:15:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:15:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:15:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:15:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:15:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:15:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:15:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:15:26.218482 2527 mini_ranger.cc:160] Time spent starting Ranger: real 8.579s user 0.000s sys 0.009s
Jun 19, 2026 8:15:45 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:15:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:15:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:15:45 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:15:46 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:15:47.326120 2527 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangerkms_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/rangerkms_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.119.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.119.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:15:47 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856947, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:15:47.428812 2527 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:15:47.439312 2527 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:15:48.372021 2527 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:15:48.372089 2527 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms
I20260619 08:15:48.372102 2527 mini_ranger_kms.cc:227] Using postgres at 127.2.119.212:58419
2026-06-19 08:15:48,547 [I] DB FLAVOR :POSTGRES
2026-06-19 08:15:48,547 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:15:48,547 [I] Checking connection
2026-06-19 08:15:48,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:15:48,809 [I] connection success
2026-06-19 08:15:48,810 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:15:48,810 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:15:48,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:15:49,056 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:15:49,056 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:15:49,056 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:15:49,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:58419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:15:49,308 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:15:49.744488 2527 mini_ranger_kms.cc:310] Created kms service
I20260619 08:15:49.966701 2527 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:15:50.011803 2527 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:15:50 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856947, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:50.423547 2527 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:15:50.423660 2527 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:15:50.423681 2527 mini_ranger_kms.cc:240] Using host: 127.2.119.212
I20260619 08:15:50.426811 2527 mini_ranger_kms.cc:276] Ranger KMS PID: 13667
I20260619 08:15:50.426975 2527 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.119.212:58081
Jun 19, 2026 8:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 08:15:52.729589 2527 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.301s user 0.002s sys 0.156s
I20260619 08:15:52.729694 2527 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:15:52.729768 2527 mini_ranger_kms.cc:401] 127.2.119.212:58081/kms/v1/keys
Jun 19 08:15:52 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781856952, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 08:15:53 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856952, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
Jun 19 08:15:53 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856952, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
Jun 19 08:15:55 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:15:55.856657 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36279
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36279
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:58081/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:55.947603 13718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:55.947818 13718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:55.947840 13718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:55.949033 13718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:15:55.949080 13718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:55.949095 13718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:55.949107 13718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:15:55.949119 13718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:15:55.950732 13718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.119.212:58081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36279
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:55.950973 13718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:55.951215 13718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:15:55.956719 13718 server_base.cc:1034] running on GCE node
W20260619 08:15:55.956737 13726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:55.956722 13723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:55.956737 13724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:55.957258 13718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:55.957576 13718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:55.958740 13718 hybrid_clock.cc:648] HybridClock initialized: now 1781856955958725 us; error 28 us; skew 500 ppm
Jun 19 08:15:55 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856955, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:55.966575 13718 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:15:55.967042 13718 webserver.cc:460] Webserver started at http://127.2.119.254:33093/ using document root <none> and password file <none>
I20260619 08:15:55.967267 13718 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:55.967306 13718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:55.967453 13718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:15:55 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856955, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:56.022061 13718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "ab528a7cfccd4a7783ade31cf0c0c853"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "bf25ef445b583f85c6b8beaeb15c47f7"
server_key_iv: "0dbd093684289fc605fdb6a9d901b6f6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.022519 13718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "ab528a7cfccd4a7783ade31cf0c0c853"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "bf25ef445b583f85c6b8beaeb15c47f7"
server_key_iv: "0dbd093684289fc605fdb6a9d901b6f6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.023942 13718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:15:56.036870 13733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:56.037092 13718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:15:56.037191 13718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ab528a7cfccd4a7783ade31cf0c0c853"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "bf25ef445b583f85c6b8beaeb15c47f7"
server_key_iv: "0dbd093684289fc605fdb6a9d901b6f6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.037354 13718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:56.044445 13718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:56.046684 13718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:56.046862 13718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:56.052214 13718 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36279
I20260619 08:15:56.052232 13785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36279 every 8 connection(s)
I20260619 08:15:56.052613 13718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:15:56.052981 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 13718
I20260619 08:15:56.053099 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:15:56.053340 2527 external_mini_cluster.cc:1428] Setting key 950fc56e717215afec9294849b766ddd
I20260619 08:15:56.053411 13786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:56.055797 13786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: Bootstrap starting.
I20260619 08:15:56.056365 13786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:56.056672 13786 log.cc:824] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:56.057868 13786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: No bootstrap required, opened a new log
I20260619 08:15:56.060076 13786 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } }
I20260619 08:15:56.060251 13786 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:56.060313 13786 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab528a7cfccd4a7783ade31cf0c0c853, State: Initialized, Role: FOLLOWER
I20260619 08:15:56.060475 13786 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } }
I20260619 08:15:56.060578 13786 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:15:56.060632 13786 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:15:56.060688 13786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:56.061441 13786 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } }
I20260619 08:15:56.061580 13786 leader_election.cc:302] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab528a7cfccd4a7783ade31cf0c0c853; no voters:
I20260619 08:15:56.061793 13786 leader_election.cc:288] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:15:56.061846 13791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:56.062074 13791 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [term 1 LEADER]: Becoming Leader. State: Replica: ab528a7cfccd4a7783ade31cf0c0c853, State: Running, Role: LEADER
I20260619 08:15:56.062196 13786 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:15:56.062219 13791 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } }
I20260619 08:15:56.062618 13791 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab528a7cfccd4a7783ade31cf0c0c853. Latest consensus state: current_term: 1 leader_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } } }
I20260619 08:15:56.062686 13791 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:56.062887 13795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 08:15:56 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:56.063999 13795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:15:56.065162 13786 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:15:56.064839 13793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab528a7cfccd4a7783ade31cf0c0c853" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36279 } } }
I20260619 08:15:56.065279 13793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853 [sys.catalog]: This master's current role is: LEADER
I20260619 08:15:56.072561 13789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:56.053866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.2.119.254:36279)
0619 08:15:56.055997 (+ 2131us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:56.056008 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:56.056045 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:15:56.056622 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:56.056631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:56.056696 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:56.056854 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:56.057528 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:56.058860 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:56.059959 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:56.060580 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:56.060686 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:56.063952 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:56.063956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:56.063956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:56.063968 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:56.068740 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:56.069239 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:56.069247 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:56.069247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:56.069623 (+ 376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:56.070982 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:56.070986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:56.070987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:56.071730 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:56.071958 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:56.072048 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 08:15:56.072345 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2080,"thread_start_us":78,"threads_started":1}
I20260619 08:15:56.074723 13795 catalog_manager.cc:1347] Generated new cluster ID: 7dcae25285e74cc6a6aa750ebf3eb832
I20260619 08:15:56.074779 13795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:15:56.105752 13795 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:15:56.106643 13795 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:15:56.111073 13795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: Generated new TSK 0
I20260619 08:15:56.111264 13795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:15:56.804394 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:58081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:56.920800 13831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:56.921011 13831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:56.921036 13831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:56.922246 13831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:56.922286 13831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:56.922313 13831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:15:56.924132 13831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.119.212:58081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:56.924366 13831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:56.924682 13831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:56.929747 13837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:56.929800 13836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:56.929980 13839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:56.930184 13831 server_base.cc:1034] running on GCE node
I20260619 08:15:56.930437 13831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:56.930716 13831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:56.931876 13831 hybrid_clock.cc:648] HybridClock initialized: now 1781856956931865 us; error 29 us; skew 500 ppm
Jun 19 08:15:56 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:56.934743 13831 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:15:56.935266 13831 webserver.cc:460] Webserver started at http://127.2.119.193:35619/ using document root <none> and password file <none>
I20260619 08:15:56.935518 13831 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:56.935583 13831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:56.935716 13831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:15:56 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:56.949949 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "e2280032d55f428a8d182bb08d79091e"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "74336cb4cae48afb0e12f97efcec4866"
server_key_iv: "b0088e46d1da8c9f66fbfcf21b40e9bd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.950475 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "e2280032d55f428a8d182bb08d79091e"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "74336cb4cae48afb0e12f97efcec4866"
server_key_iv: "b0088e46d1da8c9f66fbfcf21b40e9bd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.955268 13831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 08:15:56.973336 13848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:56.973640 13831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 08:15:56.973737 13831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "e2280032d55f428a8d182bb08d79091e"
format_stamp: "Formatted at 2026-06-19 08:15:56 on dist-test-slave-d0b5"
server_key: "74336cb4cae48afb0e12f97efcec4866"
server_key_iv: "b0088e46d1da8c9f66fbfcf21b40e9bd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:56.973829 13831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:57.000711 13831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:57.003345 13831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:57.003578 13831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:57.003885 13831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:57.004319 13831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:57.004408 13831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.004508 13831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:57.004621 13831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.012826 13831 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34669
I20260619 08:15:57.013275 13831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:15:57.013865 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 13831
I20260619 08:15:57.013963 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:15:57.014194 2527 external_mini_cluster.cc:1428] Setting key 5e19469ee0cea0d12438d354d6c6624c
I20260619 08:15:57.013650 13961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34669 every 8 connection(s)
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:57.029021 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.014521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36279 (local address 127.2.119.193:60957)
0619 08:15:57.014802 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.014845 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.015464 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.017335 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.017340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.017608 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:57.017989 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.017996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.019115 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.019116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.021751 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.021755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.022315 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.022316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.022468 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.023110 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:57.023121 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:57.024870 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:57.027191 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.027198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.027201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.027521 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.027764 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.027764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.027765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.027910 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.028246 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:57.028253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:57.028434 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.028554 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.028739 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:15:57.029675 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.014610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:60957 (local address 127.2.119.254:36279)
0619 08:15:57.017024 (+ 2414us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.017026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.017035 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:15:57.017062 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.017063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.017076 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.017154 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:57.018105 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.018998 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.021856 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.022182 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.022246 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.024987 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:57.024988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:57.024989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:57.024997 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:57.027088 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.027619 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.027621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.027621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.027691 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.027989 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.027990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.027990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.028155 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:57.028218 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.029447 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.029531 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2381,"thread_start_us":79,"threads_started":1}
I20260619 08:15:57.030454 13962 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36279
I20260619 08:15:57.030567 13962 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:57.030761 13962 heartbeater.cc:511] Master 127.2.119.254:36279 requested a full tablet report, sending...
I20260619 08:15:57.031309 13750 ts_manager.cc:194] Registered new tserver with Master: e2280032d55f428a8d182bb08d79091e (127.2.119.193:34669)
I20260619 08:15:57.032179 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:60957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:15:57.068454 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:58081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 08:15:57.190276 13970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:57.190670 13970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:57.190766 13970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:57.192461 13970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:57.192703 13970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:57.192760 13970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:15:57.195071 13970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.119.212:58081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:57.195363 13970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:57.195650 13970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:57.200932 13976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:57.200933 13978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:57.200933 13975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:57.201531 13970 server_base.cc:1034] running on GCE node
I20260619 08:15:57.201746 13970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:57.202005 13970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:57.203166 13970 hybrid_clock.cc:648] HybridClock initialized: now 1781856957203151 us; error 42 us; skew 500 ppm
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:57.206274 13970 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:15:57.206826 13970 webserver.cc:460] Webserver started at http://127.2.119.194:39843/ using document root <none> and password file <none>
I20260619 08:15:57.207100 13970 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:57.207160 13970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:57.207288 13970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:57.226670 13970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "2225f2d48bef46f99cb74107ec3f1445"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "ea9e678fad42d82d4b47e1d066fa78f0"
server_key_iv: "952abb53a0ae2edd35369e966dd6f2b6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.227097 13970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "2225f2d48bef46f99cb74107ec3f1445"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "ea9e678fad42d82d4b47e1d066fa78f0"
server_key_iv: "952abb53a0ae2edd35369e966dd6f2b6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.228605 13970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:15:57.249450 13986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.249742 13970 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260619 08:15:57.249850 13970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "2225f2d48bef46f99cb74107ec3f1445"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "ea9e678fad42d82d4b47e1d066fa78f0"
server_key_iv: "952abb53a0ae2edd35369e966dd6f2b6"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.249946 13970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:57.271938 13970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:57.274588 13970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:57.274829 13970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:57.275118 13970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:57.275564 13970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:57.275657 13970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.275728 13970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:57.275848 13970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.283742 13970 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:46407
I20260619 08:15:57.283993 14099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:46407 every 8 connection(s)
I20260619 08:15:57.284252 13970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:15:57.286062 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 13970
I20260619 08:15:57.286167 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:15:57.286360 2527 external_mini_cluster.cc:1428] Setting key c0b44da58768f207616dcbfa4cd052da
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:57.307395 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.286549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:58755 (local address 127.2.119.254:36279)
0619 08:15:57.286683 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.286685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.289044 (+ 2359us) server_negotiation.cc:407] Connection header received
0619 08:15:57.292362 (+ 3318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.292364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.292384 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.292418 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:57.293394 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.294194 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.297386 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.297722 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.297773 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.303809 (+ 6036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:57.303811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:57.303812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:57.303821 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:57.305692 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.306247 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.306248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.306249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.306309 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.306600 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.306601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.306602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.306747 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:57.306801 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.307193 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.307293 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 08:15:57.308229 14102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.285632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36279 (local address 127.2.119.194:58755)
0619 08:15:57.288872 (+ 3240us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.288928 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.289923 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.292649 (+ 2726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.292653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.292912 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:57.293217 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.293224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.294318 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.294319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.295651 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.295654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.297878 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.297879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.298034 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.301864 (+ 3830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:57.301877 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:57.303676 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:57.305805 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.305814 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.305817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.306155 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.306374 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.306374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.306375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.306523 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.306837 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:57.306845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:57.307025 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.307778 (+ 753us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.307967 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3188,"thread_start_us":74,"threads_started":1}
I20260619 08:15:57.308890 14100 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36279
I20260619 08:15:57.308982 14100 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:57.309172 14100 heartbeater.cc:511] Master 127.2.119.254:36279 requested a full tablet report, sending...
I20260619 08:15:57.309623 13750 ts_manager.cc:194] Registered new tserver with Master: 2225f2d48bef46f99cb74107ec3f1445 (127.2.119.194:46407)
I20260619 08:15:57.310122 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:58755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:15:57.344547 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:58081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:15:57.473227 14105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:15:57.473528 14105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:15:57.473568 14105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:15:57.474756 14105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:15:57.474818 14105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:15:57.474864 14105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:15:57.476390 14105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.119.212:58081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:36279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:15:57.476656 14105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:15:57.476895 14105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:15:57.486181 14113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:15:57.486182 14111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:57.487282 14105 server_base.cc:1034] running on GCE node
W20260619 08:15:57.489003 14110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:15:57.489329 14105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:15:57.489604 14105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:15:57.490763 14105 hybrid_clock.cc:648] HybridClock initialized: now 1781856957490741 us; error 37 us; skew 500 ppm
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:15:57.494197 14105 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:15:57.494767 14105 webserver.cc:460] Webserver started at http://127.2.119.195:36553/ using document root <none> and password file <none>
I20260619 08:15:57.495077 14105 fs_manager.cc:359] Metadata directory not provided
I20260619 08:15:57.495151 14105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:15:57.495368 14105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:57.514622 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "c19d68964d6845238dfae51537dba6ff"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "edd86d4951b238588eae64de01cb11b4"
server_key_iv: "77c6f24cebdcaaa342b46c013fc742cd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.515215 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "c19d68964d6845238dfae51537dba6ff"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "edd86d4951b238588eae64de01cb11b4"
server_key_iv: "77c6f24cebdcaaa342b46c013fc742cd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.517150 14105 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 08:15:57.533481 14120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.533716 14105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:15:57.533798 14105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "c19d68964d6845238dfae51537dba6ff"
format_stamp: "Formatted at 2026-06-19 08:15:57 on dist-test-slave-d0b5"
server_key: "edd86d4951b238588eae64de01cb11b4"
server_key_iv: "77c6f24cebdcaaa342b46c013fc742cd"
server_key_version: "kuduclusterkey@0"
I20260619 08:15:57.533903 14105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:15:57.567265 14105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:15:57.569720 14105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:15:57.569941 14105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:15:57.570217 14105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:15:57.570633 14105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:15:57.570725 14105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.570797 14105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:15:57.570853 14105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:15:57.578461 14105 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:41401
I20260619 08:15:57.578914 14105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:15:57.586747 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 14105
I20260619 08:15:57.586862 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:15:57.587100 2527 external_mini_cluster.cc:1428] Setting key c7f247637b981272a4844ef42be13b9e
I20260619 08:15:57.590175 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.579722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:34193 (local address 127.2.119.254:36279)
0619 08:15:57.579765 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.579767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.580048 (+ 281us) server_negotiation.cc:407] Connection header received
0619 08:15:57.580674 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.580676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.580688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.580757 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:57.581476 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.582267 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.583727 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.584042 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.584082 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.586763 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:57.586764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:57.586765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:57.586773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:57.588552 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.589041 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.589042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.589043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.589107 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.589410 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.589411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.589412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.589555 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:57.589623 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.589985 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.590080 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:15:57.590322 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.579722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36279 (local address 127.2.119.195:34193)
0619 08:15:57.579890 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.579938 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.580557 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.580827 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.580831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.581076 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:15:57.581374 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.581379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.582373 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.582374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.583628 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.583631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.584166 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.584167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.584310 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.584975 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:15:57.584985 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:15:57.586647 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:15:57.588639 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.588646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.588648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.588948 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.589180 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:15:57.589181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:15:57.589182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:15:57.589346 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:15:57.589639 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:15:57.589645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:15:57.589805 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.589920 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.590071 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260619 08:15:57.590356 14233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:41401 every 8 connection(s)
I20260619 08:15:57.590859 14234 heartbeater.cc:347] Connected to a master server at 127.2.119.254:36279
I20260619 08:15:57.590956 14234 heartbeater.cc:464] Registering TS with master...
I20260619 08:15:57.591143 14234 heartbeater.cc:511] Master 127.2.119.254:36279 requested a full tablet report, sending...
I20260619 08:15:57.591500 13750 ts_manager.cc:194] Registered new tserver with Master: c19d68964d6845238dfae51537dba6ff (127.2.119.195:41401)
I20260619 08:15:57.591998 13750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:34193
I20260619 08:15:57.598999 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:15:57.609197 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.600645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.2.119.254:36279)
0619 08:15:57.600688 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.600690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.600947 (+ 257us) server_negotiation.cc:407] Connection header received
0619 08:15:57.600976 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.600977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.600989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.601044 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:15:57.601831 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.602644 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.603747 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.604057 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.604135 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.605764 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:15:57.605765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:15:57.605766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:15:57.605774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:15:57.607524 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.607928 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.607929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.607929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.607991 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:15:57.608587 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:15:57.608588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:15:57.608588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:15:57.608731 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:15:57.608844 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.609016 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.609096 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:15:57.611568 13750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:15:57.613772 13750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:15:57.622850 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.616685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.2.119.193:34669)
0619 08:15:57.616838 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.616844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.617706 (+ 862us) server_negotiation.cc:407] Connection header received
0619 08:15:57.617790 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.617793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.617809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.617889 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:57.618150 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.619558 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.621181 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.622314 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.622365 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.622424 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.622723 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.622767 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260619 08:15:57.624729 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.616420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34669 (local address 127.0.0.1:53908)
0619 08:15:57.617582 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.617627 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.617708 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.617968 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.617972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.617977 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:57.618073 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.618075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.619640 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.619640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.621084 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.621087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.622489 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.622490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.622589 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.622604 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.622669 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.624650 (+ 1981us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":1911,"thread_start_us":86,"threads_started":1}
I20260619 08:15:57.625944 13896 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff440ea50834e31878cead910ef8dd9 (DEFAULT_TABLE table=test-table [id=cc18a531eab3473ea5f95a428ce98811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:57.626327 13896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff440ea50834e31878cead910ef8dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:57.626370 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.616895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:46407 (local address 127.0.0.1:33050)
0619 08:15:57.617922 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.617932 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.617981 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.619870 (+ 1889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.619872 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.619874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:57.619930 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.619932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.622599 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.622600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.623608 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.623610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.626132 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.626133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.626197 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.626201 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.626255 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.626300 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":79,"threads_started":1}
I20260619 08:15:57.626641 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.617188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.2.119.194:46407)
0619 08:15:57.617861 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.617865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.618038 (+ 173us) server_negotiation.cc:407] Connection header received
0619 08:15:57.618079 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.618081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.618095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.618161 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:57.620015 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.622522 (+ 2507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.623730 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.624442 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.624511 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.624586 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.626516 (+ 1930us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.626561 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":69,"threads_started":1}
I20260619 08:15:57.627108 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.617061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41401 (local address 127.0.0.1:54450)
0619 08:15:57.618264 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.618274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.618325 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.618568 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.618569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.618571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:57.618617 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.618619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.621345 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.621346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.622408 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.622411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.626873 (+ 4462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.626874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.626947 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.626952 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.627007 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.627043 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"thread_start_us":63,"threads_started":1}
I20260619 08:15:57.627220 14034 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff440ea50834e31878cead910ef8dd9 (DEFAULT_TABLE table=test-table [id=cc18a531eab3473ea5f95a428ce98811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:57.627422 14251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.617371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.2.119.195:41401)
0619 08:15:57.617535 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.617540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.618378 (+ 838us) server_negotiation.cc:407] Connection header received
0619 08:15:57.618419 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.618421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.618435 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.618498 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:57.618687 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.619771 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.625163 (+ 5392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.625953 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.626024 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.626096 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.627297 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.627340 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260619 08:15:57.627647 14034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff440ea50834e31878cead910ef8dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:57.627969 14168 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff440ea50834e31878cead910ef8dd9 (DEFAULT_TABLE table=test-table [id=cc18a531eab3473ea5f95a428ce98811]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:15:57.628235 14168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff440ea50834e31878cead910ef8dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:15:57.634495 14254 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Bootstrap starting.
I20260619 08:15:57.635466 14254 tablet_bootstrap.cc:654] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:57.635807 14254 log.cc:824] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:57.639941 14252 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Bootstrap starting.
I20260619 08:15:57.640887 14252 tablet_bootstrap.cc:654] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:57.641217 14252 log.cc:824] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:57.644764 14253 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Bootstrap starting.
I20260619 08:15:57.645738 14253 tablet_bootstrap.cc:654] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Neither blocks nor log segments found. Creating new log.
I20260619 08:15:57.646116 14253 log.cc:824] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Log is configured to *not* fsync() on all Append() calls
I20260619 08:15:57.647061 14252 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: No bootstrap required, opened a new log
I20260619 08:15:57.647133 14252 ts_tablet_manager.cc:1397] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260619 08:15:57.649534 14252 raft_consensus.cc:348] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.649907 14252 raft_consensus.cc:374] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:57.650017 14252 raft_consensus.cc:729] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2225f2d48bef46f99cb74107ec3f1445, State: Initialized, Role: FOLLOWER
I20260619 08:15:57.650243 14252 consensus_queue.cc:260] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.650655 14252 ts_tablet_manager.cc:1428] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:15:57.650827 14100 heartbeater.cc:503] Master 127.2.119.254:36279 was elected leader, sending a full tablet report...
I20260619 08:15:57.652534 14254 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: No bootstrap required, opened a new log
I20260619 08:15:57.652678 14254 ts_tablet_manager.cc:1397] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260619 08:15:57.655933 14254 raft_consensus.cc:348] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.656188 14254 raft_consensus.cc:374] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:57.656276 14254 raft_consensus.cc:729] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2280032d55f428a8d182bb08d79091e, State: Initialized, Role: FOLLOWER
I20260619 08:15:57.656469 14254 consensus_queue.cc:260] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.656818 14254 ts_tablet_manager.cc:1428] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 08:15:57.656996 13962 heartbeater.cc:503] Master 127.2.119.254:36279 was elected leader, sending a full tablet report...
I20260619 08:15:57.658095 14253 tablet_bootstrap.cc:492] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: No bootstrap required, opened a new log
I20260619 08:15:57.658237 14253 ts_tablet_manager.cc:1397] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260619 08:15:57.660363 14253 raft_consensus.cc:348] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.660705 14253 raft_consensus.cc:374] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:15:57.660811 14253 raft_consensus.cc:729] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19d68964d6845238dfae51537dba6ff, State: Initialized, Role: FOLLOWER
I20260619 08:15:57.661041 14253 consensus_queue.cc:260] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.661499 14253 ts_tablet_manager.cc:1428] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:15:57.661675 14234 heartbeater.cc:503] Master 127.2.119.254:36279 was elected leader, sending a full tablet report...
Jun 19 08:15:57 dist-test-slave-d0b5 krb5kdc[13273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:15:57.763083 14259 raft_consensus.cc:482] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:15:57.763270 14259 raft_consensus.cc:504] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.763669 14259 leader_election.cc:288] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2225f2d48bef46f99cb74107ec3f1445 (127.2.119.194:46407), c19d68964d6845238dfae51537dba6ff (127.2.119.195:41401)
W20260619 08:15:57.764526 13963 tablet.cc:2369] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:15:57.768406 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.764355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:38363 (local address 127.2.119.194:46407)
0619 08:15:57.764756 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.764759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.765003 (+ 244us) server_negotiation.cc:407] Connection header received
0619 08:15:57.765106 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.765107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.765117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.765147 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:57.765396 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.766101 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.767523 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.768023 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.768073 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.768083 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.768293 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.768336 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260619 08:15:57.768739 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.763916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:46407 (local address 127.2.119.193:38363)
0619 08:15:57.764933 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.764945 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.765032 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.765215 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.765216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.765217 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:57.765323 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.765325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.766220 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.766221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.767422 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.767425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.768141 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.768142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.768215 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.768219 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.768597 (+ 378us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.768644 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":86,"threads_started":1}
I20260619 08:15:57.769593 14054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff440ea50834e31878cead910ef8dd9" candidate_uuid: "e2280032d55f428a8d182bb08d79091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2225f2d48bef46f99cb74107ec3f1445" is_pre_election: true
I20260619 08:15:57.769747 14054 raft_consensus.cc:2454] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2280032d55f428a8d182bb08d79091e in term 0.
I20260619 08:15:57.770005 13849 leader_election.cc:302] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2225f2d48bef46f99cb74107ec3f1445, e2280032d55f428a8d182bb08d79091e; no voters:
I20260619 08:15:57.770139 14259 raft_consensus.cc:2781] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:15:57.770182 14259 raft_consensus.cc:482] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:15:57.770205 14259 raft_consensus.cc:3037] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:57.771085 14259 raft_consensus.cc:504] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.771214 14259 leader_election.cc:288] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [CANDIDATE]: Term 1 election: Requested vote from peers 2225f2d48bef46f99cb74107ec3f1445 (127.2.119.194:46407), c19d68964d6845238dfae51537dba6ff (127.2.119.195:41401)
I20260619 08:15:57.771595 14054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff440ea50834e31878cead910ef8dd9" candidate_uuid: "e2280032d55f428a8d182bb08d79091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2225f2d48bef46f99cb74107ec3f1445"
I20260619 08:15:57.771672 14054 raft_consensus.cc:3037] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:57.772586 14054 raft_consensus.cc:2454] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2280032d55f428a8d182bb08d79091e in term 1.
I20260619 08:15:57.772825 13849 leader_election.cc:302] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2225f2d48bef46f99cb74107ec3f1445, e2280032d55f428a8d182bb08d79091e; no voters:
I20260619 08:15:57.772960 14259 raft_consensus.cc:2781] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:15:57.773123 14259 raft_consensus.cc:686] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [term 1 LEADER]: Becoming Leader. State: Replica: e2280032d55f428a8d182bb08d79091e, State: Running, Role: LEADER
I20260619 08:15:57.773217 14259 consensus_queue.cc:237] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } }
I20260619 08:15:57.774053 13749 catalog_manager.cc:5495] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e reported cstate change: term changed from 0 to 1, leader changed from <none> to e2280032d55f428a8d182bb08d79091e (127.2.119.193). New cstate: current_term: 1 leader_uuid: "e2280032d55f428a8d182bb08d79091e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2280032d55f428a8d182bb08d79091e" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 } health_report { overall_health: UNKNOWN } } }
I20260619 08:15:57.775254 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.764136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:41401 (local address 127.2.119.193:57947)
0619 08:15:57.768698 (+ 4562us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.768709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:15:57.768797 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:15:57.768982 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:15:57.768982 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:15:57.768984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:15:57.769049 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.769051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.770100 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.770101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.771192 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:15:57.771194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.771967 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.771967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:15:57.772035 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.772038 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:15:57.775109 (+ 3071us) client_negotiation.cc:240] Negotiation successful
0619 08:15:57.775159 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4544,"thread_start_us":71,"threads_started":1}
I20260619 08:15:57.775256 14251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.764403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:57947 (local address 127.2.119.195:41401)
0619 08:15:57.764798 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.764800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.768769 (+ 3969us) server_negotiation.cc:407] Connection header received
0619 08:15:57.768879 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.768880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.768888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.768917 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:15:57.769126 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.769827 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.771299 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.771848 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.771898 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.771906 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.775145 (+ 3239us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.775192 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373}
I20260619 08:15:57.775842 14187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff440ea50834e31878cead910ef8dd9" candidate_uuid: "e2280032d55f428a8d182bb08d79091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19d68964d6845238dfae51537dba6ff"
I20260619 08:15:57.775822 14188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff440ea50834e31878cead910ef8dd9" candidate_uuid: "e2280032d55f428a8d182bb08d79091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19d68964d6845238dfae51537dba6ff" is_pre_election: true
I20260619 08:15:57.776008 14188 raft_consensus.cc:2454] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2280032d55f428a8d182bb08d79091e in term 0.
W20260619 08:15:57.785375 14101 tablet.cc:2369] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:15:57.814234 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.808562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53916 (local address 127.2.119.193:34669)
0619 08:15:57.808655 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:15:57.808657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:15:57.809852 (+ 1195us) server_negotiation.cc:407] Connection header received
0619 08:15:57.810038 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:15:57.810039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:15:57.810173 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:15:57.810249 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:15:57.810485 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.811267 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.812877 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:15:57.813198 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:15:57.813265 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:15:57.813493 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:15:57.813634 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:15:57.813706 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:15:57.813929 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 08:15:57.813952 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:15:57.818687 14188 raft_consensus.cc:3037] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:15:57.819763 14188 raft_consensus.cc:1261] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 1 FOLLOWER]: Refusing update from remote peer e2280032d55f428a8d182bb08d79091e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:15:57.820199 14263 consensus_queue.cc:1035] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c19d68964d6845238dfae51537dba6ff" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 41401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:15:57.822630 14054 raft_consensus.cc:1261] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 1 FOLLOWER]: Refusing update from remote peer e2280032d55f428a8d182bb08d79091e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:15:57.823017 14263 consensus_queue.cc:1035] T 9ff440ea50834e31878cead910ef8dd9 P e2280032d55f428a8d182bb08d79091e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2225f2d48bef46f99cb74107ec3f1445" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:15:57.829785 14266 mvcc.cc:205] Tried to move back new op lower bound from 7298486099222380544 to 7298486099039678464. Current Snapshot: MvccSnapshot[applied={T|T < 7298486099222380544}]
I20260619 08:15:57.830063 14267 mvcc.cc:205] Tried to move back new op lower bound from 7298486099222380544 to 7298486099039678464. Current Snapshot: MvccSnapshot[applied={T|T < 7298486099222380544}]
I20260619 08:15:57.833115 14269 mvcc.cc:205] Tried to move back new op lower bound from 7298486099222380544 to 7298486099039678464. Current Snapshot: MvccSnapshot[applied={T|T < 7298486099222380544}]
I20260619 08:15:57.840312 13749 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:15:57.840430 13749 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:15:57.841267 13749 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab528a7cfccd4a7783ade31cf0c0c853: Sending DeleteTablet for 3 replicas of tablet 9ff440ea50834e31878cead910ef8dd9
I20260619 08:15:57.841670 13896 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff440ea50834e31878cead910ef8dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:57 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:53908
I20260619 08:15:57.841895 14034 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff440ea50834e31878cead910ef8dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:57 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:33050
I20260619 08:15:57.842154 14168 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ff440ea50834e31878cead910ef8dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:15:57 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54450
I20260619 08:15:57.846081 14276 tablet_replica.cc:330] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: stopping tablet replica
I20260619 08:15:57.846199 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 13831
I20260619 08:15:57.846239 14276 raft_consensus.cc:2229] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:15:57.849349 14276 raft_consensus.cc:2258] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:15:57.849978 14276 ts_tablet_manager.cc:1905] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:15:57.851774 14277 tablet_replica.cc:330] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: stopping tablet replica
I20260619 08:15:57.851915 14277 raft_consensus.cc:2229] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:15:57.852052 14277 raft_consensus.cc:2258] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:15:57.857885 14277 ts_tablet_manager.cc:1905] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:15:57.859606 14276 ts_tablet_manager.cc:1918] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:15:57.859704 14276 log.cc:1196] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/9ff440ea50834e31878cead910ef8dd9
I20260619 08:15:57.859984 14276 ts_tablet_manager.cc:1939] T 9ff440ea50834e31878cead910ef8dd9 P 2225f2d48bef46f99cb74107ec3f1445: Deleting consensus metadata
I20260619 08:15:57.860500 13734 catalog_manager.cc:4841] TS 2225f2d48bef46f99cb74107ec3f1445 (127.2.119.194:46407): tablet 9ff440ea50834e31878cead910ef8dd9 (table test-table [id=cc18a531eab3473ea5f95a428ce98811]) successfully deleted
I20260619 08:15:57.864281 14277 ts_tablet_manager.cc:1918] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:15:57.864383 14277 log.cc:1196] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/9ff440ea50834e31878cead910ef8dd9
I20260619 08:15:57.865227 14277 ts_tablet_manager.cc:1939] T 9ff440ea50834e31878cead910ef8dd9 P c19d68964d6845238dfae51537dba6ff: Deleting consensus metadata
I20260619 08:15:57.865859 13734 catalog_manager.cc:4841] TS c19d68964d6845238dfae51537dba6ff (127.2.119.195:41401): tablet 9ff440ea50834e31878cead910ef8dd9 (table test-table [id=cc18a531eab3473ea5f95a428ce98811]) successfully deleted
W20260619 08:15:57.873726 13737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:34669 (error 108)
I20260619 08:15:57.874506 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:15:57.874375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34669 (local address 127.0.0.1:53928)
0619 08:15:57.874431 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:15:57.874468 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 08:15:57.874747 13737 catalog_manager.cc:4584] TS e2280032d55f428a8d182bb08d79091e (127.2.119.193:34669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ff440ea50834e31878cead910ef8dd9: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34669: connect: Connection refused (error 111)
I20260619 08:15:57.885444 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 13970
I20260619 08:15:57.892376 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 14105
I20260619 08:15:57.898582 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 13718
I20260619 08:15:57.905872 2527 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:15:58.250439 2527 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:15:58.250557 2527 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:15:58.595567 2527 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:15:58.595 UTC [13304] LOG: received smart shutdown request
2026-06-19 08:15:58.599 UTC [13304] LOG: background worker "logical replication launcher" (PID 13313) exited with exit code 1
2026-06-19 08:15:58.599 UTC [13308] LOG: shutting down
2026-06-19 08:15:58.783 UTC [13304] LOG: database system is shut down
2026-06-19T08:15:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46163 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:15:58 dist-test-slave-d0b5 krb5kdc[14287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:15:58 dist-test-slave-d0b5 krb5kdc[14287](info): set up 2 sockets
Jun 19 08:15:58 dist-test-slave-d0b5 krb5kdc[14287](info): commencing operation
krb5kdc: starting...
W20260619 08:15:59.934413 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:16:00 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:16:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:16:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangeradmin_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangeradmin_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangerlookup_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangerlookup_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
I20260619 08:16:00.103654 2527 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskmYJIvA/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/postgres -l logfile start
2026-06-19 08:16:02.853 UTC [14317] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 08:16:02.853 UTC [14317] LOG: listening on IPv4 address "127.2.119.212", port 48335
2026-06-19 08:16:02.857 UTC [14317] LOG: listening on Unix socket "/tmp/.s.PGSQL.48335"
2026-06-19 08:16:02.893 UTC [14320] LOG: database system was shut down at 2026-06-19 08:16:01 UTC
2026-06-19 08:16:02.898 UTC [14317] LOG: database system is ready to accept connections
I20260619 08:16:03.850219 2527 mini_postgres.cc:96] Postgres bound to 48335
2026-06-19 08:16:03.856 UTC [14328] FATAL: database "slave" does not exist
127.2.119.212:48335 - accepting connections
I20260619 08:16:03.856719 2527 mini_ranger.cc:161] Starting Ranger...
I20260619 08:16:03.871444 2527 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 08:16:04.407415 2527 mini_ranger.cc:87] Created ranger Postgres database
I20260619 08:16:04.407545 2527 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin
2026-06-19 08:16:04,672 [I] DB FLAVOR :POSTGRES
2026-06-19 08:16:04,672 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:16:04,672 [I] Checking connection..
2026-06-19 08:16:04,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 08:16:04,931 [I] Checking connection passed.
2026-06-19 08:16:04,932 [I] --------- Verifying version history table ---------
2026-06-19 08:16:04,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:16:05,189 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:16:05,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:16:05,453 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 08:16:05,453 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 08:16:05,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 08:16:05.709 UTC [14393] WARNING: there is no transaction in progress
2026-06-19 08:16:05,715 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 08:16:05,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 08:16:05,948 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 08:16:05,948 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:16:05,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 08:16:06,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 08:16:06,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-d0b5.c.gcp-upstream.internal','N') ;"
2026-06-19 08:16:06,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:16:06,945 [I] Table x_portal_user does not exist in database ranger
2026-06-19 08:16:06,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:16:07,192 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 08:16:07,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:16:07,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:16:07,731 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 08:16:07,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 08:16:08.785 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.803 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.820 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.836 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.855 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.872 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.930 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.937 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.952 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.963 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.973 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.988 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:08.997 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:09.005 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:09.012 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:09.019 UTC [14501] WARNING: there is no transaction in progress
2026-06-19 08:16:09,796 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 08:16:09,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 08:16:10,027 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 08:16:10,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 08:16:10,281 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 08:16:10,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 08:16:10,517 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 08:16:10,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 08:16:10,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 08:16:11,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-d0b5.c.gcp-upstream.internal';"
2026-06-19 08:16:11,284 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 08:16:11,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 08:16:11,547 [I] DB_PATCHES have already been applied
I20260619 08:16:11.553640 2527 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskmYJIvA/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 08:16:11.553714 2527 mini_ranger.cc:193] Using host: 127.2.119.212
I20260619 08:16:11.556792 2527 mini_ranger.cc:227] Ranger admin URL: http://127.2.119.212:56695
Jun 19, 2026 8:16:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:16:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:16:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:16:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:16:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:16:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 08:16:12.685662 2527 mini_ranger.cc:160] Time spent starting Ranger: real 8.829s user 0.000s sys 0.009s
Jun 19, 2026 8:16:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:16:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:16:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:16:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:16:33 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 08:16:34.079981 2527 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.119.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.119.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangerkms_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.119.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/rangerkms_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.119.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.119.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.119.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/HTTP_127.2.119.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 08:16:34 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856994, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 08:16:34.167812 2527 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 08:16:34.177258 2527 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 08:16:35.219069 2527 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 08:16:35.219156 2527 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms
I20260619 08:16:35.219179 2527 mini_ranger_kms.cc:227] Using postgres at 127.2.119.212:48335
2026-06-19 08:16:35,410 [I] DB FLAVOR :POSTGRES
2026-06-19 08:16:35,411 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 08:16:35,411 [I] Checking connection
2026-06-19 08:16:35,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 08:16:35,671 [I] connection success
2026-06-19 08:16:35,672 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 08:16:35,672 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 08:16:35,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 08:16:35,905 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 08:16:35,905 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 08:16:35,905 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 08:16:35,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.119.212:48335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 08:16:36,186 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 08:16:36.673777 2527 mini_ranger_kms.cc:310] Created kms service
I20260619 08:16:36.848028 2527 mini_ranger_kms.cc:326] Created kudu user
I20260619 08:16:36.889329 2527 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 08:16:36 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856994, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:37.299207 2527 mini_ranger_kms.cc:384] Added ranger policy
I20260619 08:16:37.299345 2527 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms:/tmp/dist-test-taskmYJIvA/build/release/bin/postgresql.jar:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskmYJIvA/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 08:16:37.299365 2527 mini_ranger_kms.cc:240] Using host: 127.2.119.212
I20260619 08:16:37.303014 2527 mini_ranger_kms.cc:276] Ranger KMS PID: 14681
I20260619 08:16:37.303121 2527 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.119.212:57159
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:16:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 08:16:39.616822 2527 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.449s user 0.006s sys 0.165s
I20260619 08:16:39.616928 2527 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 08:16:39.616973 2527 mini_ranger_kms.cc:401] 127.2.119.212:57159/kms/v1/keys
Jun 19 08:16:39 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781856999, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 08:16:40 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856999, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
Jun 19 08:16:40 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781856999, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.119.212@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
Jun 19 08:16:42 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:16:42.888689 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43621
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:43621
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:42.971177 14732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:42.971385 14732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:42.971408 14732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:42.972482 14732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:42.972515 14732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:42.972530 14732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:42.972543 14732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:42.972555 14732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:42.974123 14732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.119.212:57159/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:43621
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:42.974318 14732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:42.974478 14732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:42.979799 14737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:42.979815 14732 server_base.cc:1034] running on GCE node
W20260619 08:16:42.979861 14738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:42.979799 14740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:42.980402 14732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:42.980741 14732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:42.981909 14732 hybrid_clock.cc:648] HybridClock initialized: now 1781857002981886 us; error 40 us; skew 500 ppm
Jun 19 08:16:42 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:42.984447 14732 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:42.984954 14732 webserver.cc:460] Webserver started at http://127.2.119.254:41851/ using document root <none> and password file <none>
I20260619 08:16:42.985162 14732 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:42.985198 14732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:42.985287 14732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:16:42 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857002, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:43.046833 14732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "e19df7b2e7e74ee0b42bb2a2436576f2"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "b35a60bd799c05e808242f73a82904ef"
server_key_iv: "558e68945db0e6ea02bc81895766d7db"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.047240 14732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "e19df7b2e7e74ee0b42bb2a2436576f2"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "b35a60bd799c05e808242f73a82904ef"
server_key_iv: "558e68945db0e6ea02bc81895766d7db"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.048832 14732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:16:43.063616 14747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:43.063803 14732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:43.063897 14732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "e19df7b2e7e74ee0b42bb2a2436576f2"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "b35a60bd799c05e808242f73a82904ef"
server_key_iv: "558e68945db0e6ea02bc81895766d7db"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.063993 14732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:43.088420 14732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:43.090534 14732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:43.090742 14732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:43.095660 14732 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:43621
I20260619 08:16:43.095791 14799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:43621 every 8 connection(s)
I20260619 08:16:43.096119 14732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:43.096832 14800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:43.099128 14800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Bootstrap starting.
I20260619 08:16:43.099712 14800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:43.100009 14800 log.cc:824] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:43.100685 14800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: No bootstrap required, opened a new log
I20260619 08:16:43.103001 14800 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:43.103168 14800 raft_consensus.cc:374] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:43.103230 14800 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19df7b2e7e74ee0b42bb2a2436576f2, State: Initialized, Role: FOLLOWER
I20260619 08:16:43.103415 14800 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:43.103516 14800 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:43.103559 14800 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:43.103606 14800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:43.104167 14800 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:43.104285 14800 leader_election.cc:302] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19df7b2e7e74ee0b42bb2a2436576f2; no voters:
I20260619 08:16:43.104461 14800 leader_election.cc:288] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:16:43.104494 14803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:43.104676 14803 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 LEADER]: Becoming Leader. State: Replica: e19df7b2e7e74ee0b42bb2a2436576f2, State: Running, Role: LEADER
I20260619 08:16:43.104712 14800 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:43.104803 14803 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:43.105129 14803 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19df7b2e7e74ee0b42bb2a2436576f2. Latest consensus state: current_term: 1 leader_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } } }
I20260619 08:16:43.105237 14803 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:43.105154 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } } }
I20260619 08:16:43.105338 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:43.105554 14812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:43.106052 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 14732
I20260619 08:16:43.106123 14812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:43.106176 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:16:43.106415 2527 external_mini_cluster.cc:1428] Setting key 99704a9753b62fc2220e055982032ec5
I20260619 08:16:43.106470 14800 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:16:43.111212 14812 catalog_manager.cc:1347] Generated new cluster ID: 3158b77d6450402f866f70c7872760ea
I20260619 08:16:43.111341 14812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:16:43 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:43.121645 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:43.106862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.2.119.254:43621)
0619 08:16:43.109560 (+ 2698us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:43.109574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:43.109644 (+ 70us) server_negotiation.cc:407] Connection header received
0619 08:16:43.109929 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:43.109937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:43.110013 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:43.110177 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:43.110987 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:43.112500 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:43.113662 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:43.114320 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:43.114410 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:43.117679 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:43.117683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:43.117684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:43.117696 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:43.119794 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:43.120229 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:43.120232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:43.120233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:43.120333 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:43.120547 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:43.120548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:43.120548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:43.120861 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:43.120997 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:43.121261 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 08:16:43.121409 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2579,"thread_start_us":1776,"threads_started":1}
I20260619 08:16:43.130673 14812 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:16:43.131428 14812 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:43.145166 14812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Generated new TSK 0
I20260619 08:16:43.145396 14812 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:16:43.781468 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:43.902733 14845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:43.903033 14845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:43.903067 14845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:43.904758 14845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:43.904829 14845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:43.904870 14845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:16:43.907048 14845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:43.907297 14845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:43.907577 14845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:43.913028 14853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:43.913022 14851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:43.913131 14850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:43.913566 14845 server_base.cc:1034] running on GCE node
I20260619 08:16:43.913774 14845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:43.914022 14845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:43.915191 14845 hybrid_clock.cc:648] HybridClock initialized: now 1781857003915176 us; error 31 us; skew 500 ppm
Jun 19 08:16:43 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857003, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:43.917775 14845 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:16:43.918254 14845 webserver.cc:460] Webserver started at http://127.2.119.193:45283/ using document root <none> and password file <none>
I20260619 08:16:43.918488 14845 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:43.918556 14845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:43.918708 14845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:16:43 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857003, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:43.935779 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "390007f98f4abfeabadfc19a8164526a"
server_key_iv: "bd3bbac022f23d521283299778582b35"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.936329 14845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "390007f98f4abfeabadfc19a8164526a"
server_key_iv: "bd3bbac022f23d521283299778582b35"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.941810 14845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 08:16:43.956036 14860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:43.956267 14845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:43.956358 14845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "390007f98f4abfeabadfc19a8164526a"
server_key_iv: "bd3bbac022f23d521283299778582b35"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:43.956462 14845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:43.994776 14845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:43.997241 14845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:43.997498 14845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:43.997828 14845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:43.998248 14845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:43.998306 14845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:43.998382 14845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:43.998432 14845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.006500 14845 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:43679
I20260619 08:16:44.006938 14845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:16:44.007599 14975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:43679 every 8 connection(s)
I20260619 08:16:44.010027 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 14845
I20260619 08:16:44.010136 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:16:44.010360 2527 external_mini_cluster.cc:1428] Setting key 132a2dd3a56095c090f5ebb0ab4e7840
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857003, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:44.018987 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.008109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:34771 (local address 127.2.119.254:43621)
0619 08:16:44.008290 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:44.008292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:44.008304 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:16:44.008760 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:44.008761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:44.008777 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:44.008838 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:44.009603 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.010470 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.012183 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.012571 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.012644 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.015333 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:44.015335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:44.015335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:44.015343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:44.017040 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.017660 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.017661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.017662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.017736 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.018082 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.018083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.018084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.018251 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:44.018324 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:44.018771 (+ 447us) server_negotiation.cc:299] Negotiation successful
0619 08:16:44.018879 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
I20260619 08:16:44.019160 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.007770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.193:34771)
0619 08:16:44.007962 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:44.008004 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:44.008634 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:44.008980 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:44.008983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:44.009191 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:44.009491 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.009496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.010593 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.010594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.012074 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.012077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.012735 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.012736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.012906 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.013636 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:44.013649 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:44.015225 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:44.017152 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.017159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.017163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.017559 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.017834 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.017835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.017836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.017994 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.018356 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:44.018364 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:44.018560 (+ 196us) client_negotiation.cc:769] Sending connection context
0619 08:16:44.018690 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 08:16:44.018890 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260619 08:16:44.019886 14976 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:44.020004 14976 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.020215 14976 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:44.020829 14764 ts_manager.cc:194] Registered new tserver with Master: 72ecb70bacb24b3cb2a5f79466f4a4bd (127.2.119.193:43679)
I20260619 08:16:44.021750 14764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:34771
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:16:44.045085 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 08:16:44.162011 14982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:44.162294 14982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:44.162338 14982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:44.164243 14982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:44.164314 14982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:44.164357 14982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:16:44.166635 14982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:44.167047 14982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:44.167343 14982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:44.173743 14990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.174065 14989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.174207 14992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:44.175321 14982 server_base.cc:1034] running on GCE node
I20260619 08:16:44.175540 14982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:44.175837 14982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:44.183208 14982 hybrid_clock.cc:648] HybridClock initialized: now 1781857004183177 us; error 53 us; skew 500 ppm
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:44.186311 14982 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:16:44.186831 14982 webserver.cc:460] Webserver started at http://127.2.119.194:33495/ using document root <none> and password file <none>
I20260619 08:16:44.187107 14982 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:44.187161 14982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:44.187289 14982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:44.202926 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "688d643ca9a242e590382a1bf2049fc3"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "a7055fe2343760cdce669dd6c0b14b2a"
server_key_iv: "d73d2c5a52a2e0b5e0a0263c6d923a1c"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.203379 14982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "688d643ca9a242e590382a1bf2049fc3"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "a7055fe2343760cdce669dd6c0b14b2a"
server_key_iv: "d73d2c5a52a2e0b5e0a0263c6d923a1c"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.205999 14982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 08:16:44.225440 14999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.225678 14982 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260619 08:16:44.225772 14982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "688d643ca9a242e590382a1bf2049fc3"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "a7055fe2343760cdce669dd6c0b14b2a"
server_key_iv: "d73d2c5a52a2e0b5e0a0263c6d923a1c"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.225860 14982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:44.266579 14982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:44.269187 14982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:44.269433 14982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:44.269701 14982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:44.270140 14982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:44.270244 14982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.270355 14982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:44.270421 14982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.276889 14982 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:37029
I20260619 08:16:44.277428 14982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:16:44.282115 15113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:37029 every 8 connection(s)
I20260619 08:16:44.286233 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 14982
I20260619 08:16:44.286350 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:16:44.286612 2527 external_mini_cluster.cc:1428] Setting key 8d2f75c81e1d4ae7e44cb7fcea9b6100
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
I20260619 08:16:44.302572 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.283093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:50789 (local address 127.2.119.254:43621)
0619 08:16:44.283144 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:44.283146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:44.285621 (+ 2475us) server_negotiation.cc:407] Connection header received
0619 08:16:44.286334 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:44.286335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:44.286351 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:44.286433 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:44.287194 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.287926 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.289521 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.289874 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.289920 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.292838 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:44.292839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:44.292839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:44.292848 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:44.300073 (+ 7225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.301278 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.301279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.301280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.301373 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.301677 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.301678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.301678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.301839 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:44.301902 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:44.302319 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 08:16:44.302437 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:44.303185 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.282788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.194:50789)
0619 08:16:44.285440 (+ 2652us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:44.285496 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:44.286192 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:44.286541 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:44.286546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:44.286807 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:44.287092 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.287098 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.288045 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.288046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.289416 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.289419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.290232 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.290233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.290387 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.290964 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:44.290974 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:44.292715 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:44.300783 (+ 8068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.300796 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.300799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.301180 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.301449 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.301450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.301450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.301603 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.301957 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:44.301966 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:44.302160 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 08:16:44.302725 (+ 565us) client_negotiation.cc:240] Negotiation successful
0619 08:16:44.302914 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2598,"thread_start_us":99,"threads_started":1}
I20260619 08:16:44.303740 15114 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:44.303833 15114 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.304031 15114 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:44.304500 14764 ts_manager.cc:194] Registered new tserver with Master: 688d643ca9a242e590382a1bf2049fc3 (127.2.119.194:37029)
I20260619 08:16:44.304890 14764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:50789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:16:44.327963 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:44.468948 15119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:44.469286 15119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:44.469406 15119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:44.471113 15119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:44.471230 15119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:44.471316 15119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:16:44.473552 15119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:44.474021 15119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:44.474350 15119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:44.481113 15124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.481442 15125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.481524 15127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:44.482585 15119 server_base.cc:1034] running on GCE node
I20260619 08:16:44.482889 15119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:44.483145 15119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:44.484318 15119 hybrid_clock.cc:648] HybridClock initialized: now 1781857004484298 us; error 32 us; skew 500 ppm
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:44.487649 15119 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:16:44.488379 15119 webserver.cc:460] Webserver started at http://127.2.119.195:39123/ using document root <none> and password file <none>
I20260619 08:16:44.488811 15119 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:44.488929 15119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:44.489104 15119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:44.511178 15119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "c361d8f61133407581a313ee2506921c"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "59bda0bbb0381b1a34077633b9976b6e"
server_key_iv: "cfad21618fcb268c38df6abb581d84d4"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.511759 15119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "c361d8f61133407581a313ee2506921c"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "59bda0bbb0381b1a34077633b9976b6e"
server_key_iv: "cfad21618fcb268c38df6abb581d84d4"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.517717 15119 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260619 08:16:44.532985 15134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.533233 15119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:44.533349 15119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "c361d8f61133407581a313ee2506921c"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "59bda0bbb0381b1a34077633b9976b6e"
server_key_iv: "cfad21618fcb268c38df6abb581d84d4"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.533440 15119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:44.552651 15119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:44.555269 15119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:44.555487 15119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:44.555778 15119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:44.556206 15119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:44.556291 15119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.556376 15119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:44.556435 15119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.562835 15119 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:37379
I20260619 08:16:44.563313 15119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:16:44.566138 15247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:37379 every 8 connection(s)
I20260619 08:16:44.571516 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15119
I20260619 08:16:44.571605 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:16:44.571795 2527 external_mini_cluster.cc:1428] Setting key 73978a919a1231301e2d5c1993bd4144
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:44.578626 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.567075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:39203 (local address 127.2.119.254:43621)
0619 08:16:44.567125 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:44.567127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:44.567605 (+ 478us) server_negotiation.cc:407] Connection header received
0619 08:16:44.568669 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:44.568670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:44.568683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:44.568714 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:44.569656 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.570464 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.571955 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.572252 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.572294 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.575149 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:44.575150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:44.575150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:44.575158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:44.576924 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.577499 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.577500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.577501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.577562 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.577854 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.577855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.577856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.577997 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:44.578054 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:44.578442 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 08:16:44.578529 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:16:44.579182 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.566745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.195:39203)
0619 08:16:44.567422 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:44.567484 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:44.568547 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:44.568897 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:44.568901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:44.569231 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:44.569543 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.569550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.570579 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.570579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.571849 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.571852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.572423 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.572424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.572560 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.573172 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:44.573182 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:44.575038 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:44.577052 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.577058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.577061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.577390 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.577637 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.577638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.577638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.577769 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.578090 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:44.578096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:44.578261 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 08:16:44.578783 (+ 522us) client_negotiation.cc:240] Negotiation successful
0619 08:16:44.578944 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":84,"threads_started":1}
I20260619 08:16:44.579871 15248 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:44.579967 15248 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.580158 15248 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:44.580575 14764 ts_manager.cc:194] Registered new tserver with Master: c361d8f61133407581a313ee2506921c (127.2.119.195:37379)
I20260619 08:16:44.581099 14764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:39203
I20260619 08:16:44.583389 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:16:44.584175 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 14845
I20260619 08:16:44.589782 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 14982
I20260619 08:16:44.594488 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15119
I20260619 08:16:44.603801 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 14732
I20260619 08:16:44.610551 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43621
--webserver_interface=127.2.119.254
--webserver_port=41851
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:43621
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:44.694089 15251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:44.694330 15251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:44.694367 15251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:44.695540 15251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:44.695609 15251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:44.695636 15251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:44.695664 15251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:44.695691 15251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:44.697409 15251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:43621
--ranger_config_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:44.697674 15251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:44.697902 15251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:16:44.703837 15251 server_base.cc:1034] running on GCE node
W20260619 08:16:44.703804 15257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.703934 15259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.703819 15256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:44.704238 15251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:44.704586 15251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:44.705749 15251 hybrid_clock.cc:648] HybridClock initialized: now 1781857004705727 us; error 35 us; skew 500 ppm
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:44.708873 15251 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:44.709524 15251 webserver.cc:460] Webserver started at http://127.2.119.254:41851/ using document root <none> and password file <none>
I20260619 08:16:44.709784 15251 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:44.709836 15251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:44.750674 15251 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:44.751439 15267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.751631 15251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:44.751742 15251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "e19df7b2e7e74ee0b42bb2a2436576f2"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "b35a60bd799c05e808242f73a82904ef"
server_key_iv: "558e68945db0e6ea02bc81895766d7db"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.752252 15251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:44.782071 15251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:44.784176 15251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:44.784361 15251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:44.789481 15251 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:43621
I20260619 08:16:44.789557 15319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:43621 every 8 connection(s)
I20260619 08:16:44.789937 15251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:44.791555 15320 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:16:44.792384 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Bootstrap starting.
I20260619 08:16:44.797616 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15251
I20260619 08:16:44.797948 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:43679
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:43621
--webserver_port=45283
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:44.804597 15320 log.cc:824] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:44.806998 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:16:44.807361 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Bootstrap complete.
I20260619 08:16:44.810770 15320 raft_consensus.cc:348] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:44.811139 15320 raft_consensus.cc:729] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19df7b2e7e74ee0b42bb2a2436576f2, State: Initialized, Role: FOLLOWER
I20260619 08:16:44.811350 15320 consensus_queue.cc:260] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:44.811440 15320 raft_consensus.cc:388] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:44.811517 15320 raft_consensus.cc:482] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:44.811576 15320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:16:44.812956 15320 raft_consensus.cc:504] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:44.813071 15320 leader_election.cc:302] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e19df7b2e7e74ee0b42bb2a2436576f2; no voters:
I20260619 08:16:44.813269 15320 leader_election.cc:288] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:16:44.813359 15324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:16:44.813568 15320 sys_catalog.cc:564] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:44.813606 15324 raft_consensus.cc:686] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [term 2 LEADER]: Becoming Leader. State: Replica: e19df7b2e7e74ee0b42bb2a2436576f2, State: Running, Role: LEADER
I20260619 08:16:44.813894 15324 consensus_queue.cc:237] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } }
I20260619 08:16:44.814215 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } } }
I20260619 08:16:44.814284 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:44.814553 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e19df7b2e7e74ee0b42bb2a2436576f2. Latest consensus state: current_term: 2 leader_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e19df7b2e7e74ee0b42bb2a2436576f2" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 43621 } } }
I20260619 08:16:44.814646 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:44.814965 15332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:44.815279 15320 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 08:16:44.815472 15332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:44.815605 15332 catalog_manager.cc:1259] Loaded cluster ID: 3158b77d6450402f866f70c7872760ea
I20260619 08:16:44.815651 15332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:16:44.819742 15332 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:44.820420 15332 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Loaded TSK: 0
I20260619 08:16:44.820729 15332 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:16:44.909235 15322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:44.909531 15322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:44.909572 15322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:44.910845 15322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:44.910915 15322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:44.910959 15322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:16:44.912480 15322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:43679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=45283
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:44.912743 15322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:44.913009 15322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:44.918799 15361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:44.921558 15358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:44.923519 15322 server_base.cc:1034] running on GCE node
W20260619 08:16:44.923792 15359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:44.924103 15322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:44.924404 15322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:44.925554 15322 hybrid_clock.cc:648] HybridClock initialized: now 1781857004925542 us; error 28 us; skew 500 ppm
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:44.928383 15322 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:16:44.928907 15322 webserver.cc:460] Webserver started at http://127.2.119.193:45283/ using document root <none> and password file <none>
I20260619 08:16:44.929162 15322 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:44.929278 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:44.948159 15322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:44.949256 15369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.949534 15322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:44.949630 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd"
format_stamp: "Formatted at 2026-06-19 08:16:43 on dist-test-slave-d0b5"
server_key: "390007f98f4abfeabadfc19a8164526a"
server_key_iv: "bd3bbac022f23d521283299778582b35"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:44.950089 15322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:44.965035 15322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:44.967562 15322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:44.967715 15322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:44.967970 15322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:44.968353 15322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:44.968397 15322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.968429 15322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:44.968477 15322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:44.976429 15322 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:43679
I20260619 08:16:44.976583 15482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:43679 every 8 connection(s)
I20260619 08:16:44.976975 15322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:16:44 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857004, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:44.985489 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15322
I20260619 08:16:44.985800 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:37029
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:43621
--webserver_port=33495
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:44.992292 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.977859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54683 (local address 127.2.119.254:43621)
0619 08:16:44.978078 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:44.978088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:44.978349 (+ 261us) server_negotiation.cc:407] Connection header received
0619 08:16:44.979091 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:44.979099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:44.979157 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:44.979302 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:44.980063 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.981557 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.982858 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.983333 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.983450 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.987768 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:44.987773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:44.987773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:44.987787 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:44.990030 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.990638 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.990643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.990644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.990776 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:44.991115 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:44.991116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:44.991116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:44.991437 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:44.991529 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:44.991933 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 08:16:44.992066 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260619 08:16:44.992362 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:44.977773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.193:54683)
0619 08:16:44.978170 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:44.978213 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:44.978842 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:44.979374 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:44.979380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:44.979635 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:44.979928 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.979935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.981677 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.981678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.982752 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:44.982756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:44.983455 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:44.983457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:44.983602 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:44.984192 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:44.984201 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:44.987615 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:44.990155 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.990163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.990165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.990536 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.990865 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:44.990866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:44.990867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:44.991030 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:44.991525 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:44.991532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:44.991720 (+ 188us) client_negotiation.cc:769] Sending connection context
0619 08:16:44.991853 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 08:16:44.992048 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":81,"threads_started":1}
I20260619 08:16:44.993257 15483 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:44.993393 15483 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.993613 15483 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
W20260619 08:16:44.994607 15483 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:43621 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:16:44.994678 15483 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.994750 15483 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:44.995193 15483 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:44.995270 15483 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
W20260619 08:16:44.995723 15483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:16:45.092532 15487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:45.092859 15487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:45.092923 15487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:45.094660 15487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:45.094733 15487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:45.094779 15487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:16:45.096940 15487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:37029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=33495
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:45.097205 15487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:45.097523 15487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:45.103338 15493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:45.103425 15492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:45.103473 15495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:45.104020 15487 server_base.cc:1034] running on GCE node
I20260619 08:16:45.104380 15487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:45.104753 15487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:45.105921 15487 hybrid_clock.cc:648] HybridClock initialized: now 1781857005105889 us; error 47 us; skew 500 ppm
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:45.109241 15487 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:16:45.109836 15487 webserver.cc:460] Webserver started at http://127.2.119.194:33495/ using document root <none> and password file <none>
I20260619 08:16:45.110119 15487 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:45.110198 15487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:45.126917 15487 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260619 08:16:45.130391 15503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.130584 15487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260619 08:16:45.130668 15487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "688d643ca9a242e590382a1bf2049fc3"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "a7055fe2343760cdce669dd6c0b14b2a"
server_key_iv: "d73d2c5a52a2e0b5e0a0263c6d923a1c"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:45.131150 15487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:45.162766 15487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:45.164839 15487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:45.165012 15487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:45.165253 15487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:45.165664 15487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:45.165716 15487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.165761 15487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:45.165784 15487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.172904 15487 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:37029
I20260619 08:16:45.173038 15616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:37029 every 8 connection(s)
I20260619 08:16:45.173385 15487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:45.182641 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15487
I20260619 08:16:45.182976 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.119.212:57159/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:37379
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:43621
--webserver_port=39123
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:32803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:45.184517 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:45.174131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:37495 (local address 127.2.119.254:43621)
0619 08:16:45.174205 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:45.174209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:45.174494 (+ 285us) server_negotiation.cc:407] Connection header received
0619 08:16:45.175156 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:45.175157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:45.175170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:45.175201 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:45.175939 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.176454 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.178340 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.178568 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.178636 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:45.180982 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:45.180984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:45.180985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:45.180993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:45.182881 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.183412 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.183413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.183414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.183484 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.183755 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.183756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.183757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.183891 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:45.183932 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:45.184302 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 08:16:45.184401 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 08:16:45.185067 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:45.174114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.194:37495)
0619 08:16:45.174327 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:45.174378 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:45.175031 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:45.175301 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:45.175305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:45.175537 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:45.175832 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:45.175840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.176558 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.176559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:45.178241 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:45.178244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.178676 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.178677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:45.178792 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:45.179329 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:45.179338 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:45.180837 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:45.182994 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:45.183000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:45.183002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:45.183316 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:45.183556 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:45.183557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:45.183557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:45.183687 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:45.183977 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:45.183983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:45.184146 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 08:16:45.184679 (+ 533us) client_negotiation.cc:240] Negotiation successful
0619 08:16:45.184833 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260619 08:16:45.185559 15617 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:45.185668 15617 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.185847 15617 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
W20260619 08:16:45.186486 15617 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:43621 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:16:45.186563 15617 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.186631 15617 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:45.186903 15617 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.186980 15617 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
W20260619 08:16:45.187245 15617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:16:45.307047 15620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:45.307396 15620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:45.307476 15620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:45.309185 15620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:45.309340 15620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:45.309434 15620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:16:45.311690 15620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:32803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.119.212:57159/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:37379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=39123
--tserver_master_addrs=127.2.119.254:43621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:45.312096 15620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:45.312397 15620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:45.318090 15627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:45.318843 15626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:45.319058 15629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:45.319587 15620 server_base.cc:1034] running on GCE node
I20260619 08:16:45.319813 15620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:45.320107 15620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:45.321476 15620 hybrid_clock.cc:648] HybridClock initialized: now 1781857005321456 us; error 35 us; skew 500 ppm
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:45.326045 15620 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:16:45.326586 15620 webserver.cc:460] Webserver started at http://127.2.119.195:39123/ using document root <none> and password file <none>
I20260619 08:16:45.326900 15620 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:45.327001 15620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:45.349635 15620 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 08:16:45.350421 15637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.350633 15620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:16:45.350760 15620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "c361d8f61133407581a313ee2506921c"
format_stamp: "Formatted at 2026-06-19 08:16:44 on dist-test-slave-d0b5"
server_key: "59bda0bbb0381b1a34077633b9976b6e"
server_key_iv: "cfad21618fcb268c38df6abb581d84d4"
server_key_version: "kuduclusterkey@0"
I20260619 08:16:45.353014 15620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:45.407899 15620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:45.411840 15620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:45.412024 15620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:45.412281 15620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:45.412717 15620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:45.412814 15620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.412886 15620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:45.412940 15620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:45.420428 15620 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:37379
I20260619 08:16:45.420513 15750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:37379 every 8 connection(s)
I20260619 08:16:45.420892 15620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:16:45.422860 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15620
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:45.431769 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:45.421705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:41395 (local address 127.2.119.254:43621)
0619 08:16:45.421778 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:45.421780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:45.422003 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:16:45.422630 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:45.422631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:45.422644 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:45.422717 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:45.423483 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.424266 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.425393 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.425689 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.425768 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:45.428171 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:45.428172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:45.428173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:45.428181 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:45.430230 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.430734 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.430735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.430735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.430794 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.431080 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.431081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.431081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.431229 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:45.431296 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:45.431634 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 08:16:45.431702 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:16:45.432029 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:45.421696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:43621 (local address 127.2.119.195:41395)
0619 08:16:45.421869 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:45.421917 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:45.422492 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:45.422816 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:45.422821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:45.423065 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:45.423394 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:45.423400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.424369 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.424371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:45.425253 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:45.425255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.425808 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.425809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:45.425951 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:45.426514 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:45.426523 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:45.428068 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:45.430324 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:45.430333 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:45.430335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:45.430653 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:45.430866 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:45.430867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:45.430867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:45.431009 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:45.431305 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:45.431312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:45.431483 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 08:16:45.431612 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 08:16:45.431774 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260619 08:16:45.432550 15751 heartbeater.cc:347] Connected to a master server at 127.2.119.254:43621
I20260619 08:16:45.432655 15751 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.432861 15751 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:45.433099 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:45.423643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.2.119.254:43621)
0619 08:16:45.423764 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:45.423766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:45.423774 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:16:45.423832 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:45.423832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:45.423844 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:45.423933 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:45.425378 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.425965 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.427074 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:45.427306 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:45.427394 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:45.428750 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:45.428751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:45.428752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:45.428758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:45.430525 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.430948 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.430949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.430949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.430994 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:45.431199 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:45.431200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:45.431200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:45.431321 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:45.431439 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:45.432908 (+ 1469us) server_negotiation.cc:299] Negotiation successful
0619 08:16:45.432994 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
W20260619 08:16:45.433974 15751 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:43621 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 08:16:45.434051 15751 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.434118 15751 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:45.434500 15751 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.434578 15751 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
W20260619 08:16:45.434875 15751 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 08:16:45 dist-test-slave-d0b5 krb5kdc[14287](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:45.995944 15483 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:45.996111 15483 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:45.996551 15284 ts_manager.cc:194] Registered new tserver with Master: 72ecb70bacb24b3cb2a5f79466f4a4bd (127.2.119.193:43679)
I20260619 08:16:45.997602 15284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:54683
I20260619 08:16:46.187454 15617 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:46.187633 15617 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:46.187976 15284 ts_manager.cc:194] Registered new tserver with Master: 688d643ca9a242e590382a1bf2049fc3 (127.2.119.194:37029)
I20260619 08:16:46.188522 15284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:37495
Jun 19 08:16:46 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:46.435098 15751 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:46.435288 15751 heartbeater.cc:511] Master 127.2.119.254:43621 requested a full tablet report, sending...
I20260619 08:16:46.435659 15284 ts_manager.cc:194] Registered new tserver with Master: c361d8f61133407581a313ee2506921c (127.2.119.195:37379)
I20260619 08:16:46.436228 15284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:41395
I20260619 08:16:46.439098 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:16:46.448122 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.440081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.2.119.254:43621)
0619 08:16:46.440343 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.440346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.440353 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:16:46.440375 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.440376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.440387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.440469 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:46.441209 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.442042 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.443021 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.443336 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.443416 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.444942 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:46.444944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:46.444944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:46.444952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:46.446729 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:46.447135 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:46.447136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:46.447137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:46.447204 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:46.447430 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:46.447431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:46.447431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:46.447565 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:46.447669 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.447814 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.447913 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":68,"threads_started":1}
I20260619 08:16:46.450663 15284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:16:46.453436 15284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:16:46.464828 15780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43679 (local address 127.0.0.1:37514)
0619 08:16:46.457322 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.457337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.457405 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.459050 (+ 1645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:46.459053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:46.459055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:46.459197 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.459199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.460507 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.460508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.462451 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.462454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.464504 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.464504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.464598 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.464612 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:16:46.464678 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 08:16:46.464733 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":103,"threads_started":1}
I20260619 08:16:46.465365 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.2.119.193:43679)
0619 08:16:46.457050 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.457055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.457520 (+ 465us) server_negotiation.cc:407] Connection header received
0619 08:16:46.457562 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.457565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.457582 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.457656 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:46.459290 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.460409 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.462558 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.463382 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.463468 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.463533 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.465215 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.465266 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":50,"threads_started":1}
I20260619 08:16:46.465469 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.2.119.194:37029)
0619 08:16:46.461700 (+ 4844us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.461704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.461713 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:16:46.461741 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.461743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.461756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.461831 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:46.461996 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.463000 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.464243 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.465007 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.465077 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.465137 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.465352 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.465394 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4810,"thread_start_us":53,"threads_started":1}
I20260619 08:16:46.465725 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:37029 (local address 127.0.0.1:39914)
0619 08:16:46.457771 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.457792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.457838 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.461867 (+ 4029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:46.461868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:46.461869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:46.461922 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.461923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.463101 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.463102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.464147 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.464149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.465169 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.465170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.465234 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.465238 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:46.465281 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:16:46.465654 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":64,"threads_started":1}
I20260619 08:16:46.468201 15551 tablet_service.cc:1467] Processing CreateTablet for tablet fd25f674868a4b248fc205def9d0a71c (DEFAULT_TABLE table=test-table [id=89579056627c46b8bf2d17cd501755b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:46.468215 15417 tablet_service.cc:1467] Processing CreateTablet for tablet fd25f674868a4b248fc205def9d0a71c (DEFAULT_TABLE table=test-table [id=89579056627c46b8bf2d17cd501755b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:46.468575 15551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd25f674868a4b248fc205def9d0a71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:46.468565 15417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd25f674868a4b248fc205def9d0a71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:46.471266 15785 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: Bootstrap starting.
I20260619 08:16:46.472321 15785 tablet_bootstrap.cc:654] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:46.472694 15785 log.cc:824] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:46.473260 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37379 (local address 127.0.0.1:40552)
0619 08:16:46.457119 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.458918 (+ 1799us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.458976 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.461735 (+ 2759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:46.461736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:46.461738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:46.461790 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.461792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.466205 (+ 4413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.466206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.467315 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.467318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.473010 (+ 5692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.473011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.473094 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.473099 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:16:46.473129 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 08:16:46.473174 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":78,"threads_started":1}
I20260619 08:16:46.473537 15785 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: No bootstrap required, opened a new log
I20260619 08:16:46.473605 15785 ts_tablet_manager.cc:1397] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:16:46.473889 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.456898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.2.119.195:37379)
0619 08:16:46.461417 (+ 4519us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.461422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.461433 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:16:46.461467 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.461469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.461484 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.461561 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:46.463574 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.466109 (+ 2535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.472078 (+ 5969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.472890 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.473645 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.473722 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.473751 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.473810 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4487,"thread_start_us":49,"threads_started":1}
I20260619 08:16:46.474242 15786 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: Bootstrap starting.
I20260619 08:16:46.474562 15685 tablet_service.cc:1467] Processing CreateTablet for tablet fd25f674868a4b248fc205def9d0a71c (DEFAULT_TABLE table=test-table [id=89579056627c46b8bf2d17cd501755b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:46.474843 15786 tablet_bootstrap.cc:654] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:46.474864 15685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd25f674868a4b248fc205def9d0a71c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:46.475113 15786 log.cc:824] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:46.476107 15785 raft_consensus.cc:348] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.476271 15785 raft_consensus.cc:374] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:46.476307 15785 raft_consensus.cc:729] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688d643ca9a242e590382a1bf2049fc3, State: Initialized, Role: FOLLOWER
I20260619 08:16:46.476545 15785 consensus_queue.cc:260] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.476881 15785 ts_tablet_manager.cc:1428] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:16:46.477238 15617 heartbeater.cc:503] Master 127.2.119.254:43621 was elected leader, sending a full tablet report...
I20260619 08:16:46.478525 15790 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Bootstrap starting.
I20260619 08:16:46.479197 15790 tablet_bootstrap.cc:654] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:46.479555 15790 log.cc:824] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:46.479686 15786 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: No bootstrap required, opened a new log
I20260619 08:16:46.479758 15786 ts_tablet_manager.cc:1397] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260619 08:16:46.480430 15790 tablet_bootstrap.cc:492] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: No bootstrap required, opened a new log
I20260619 08:16:46.480504 15790 ts_tablet_manager.cc:1397] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:16:46.482846 15786 raft_consensus.cc:348] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.483006 15786 raft_consensus.cc:374] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:46.483042 15786 raft_consensus.cc:729] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ecb70bacb24b3cb2a5f79466f4a4bd, State: Initialized, Role: FOLLOWER
I20260619 08:16:46.483196 15786 consensus_queue.cc:260] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.483462 15786 ts_tablet_manager.cc:1428] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260619 08:16:46.483384 15790 raft_consensus.cc:348] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.483516 15790 raft_consensus.cc:374] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:46.483551 15790 raft_consensus.cc:729] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c361d8f61133407581a313ee2506921c, State: Initialized, Role: FOLLOWER
I20260619 08:16:46.483572 15483 heartbeater.cc:503] Master 127.2.119.254:43621 was elected leader, sending a full tablet report...
I20260619 08:16:46.484122 15790 consensus_queue.cc:260] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.484510 15790 ts_tablet_manager.cc:1428] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 08:16:46.484669 15751 heartbeater.cc:503] Master 127.2.119.254:43621 was elected leader, sending a full tablet report...
Jun 19 08:16:46 dist-test-slave-d0b5 krb5kdc[14287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857005, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for HTTP/127.2.119.212@KRBTEST.COM
I20260619 08:16:46.604961 15793 raft_consensus.cc:482] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:16:46.605106 15793 raft_consensus.cc:504] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.605546 15793 leader_election.cc:288] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72ecb70bacb24b3cb2a5f79466f4a4bd (127.2.119.193:43679), 688d643ca9a242e590382a1bf2049fc3 (127.2.119.194:37029)
I20260619 08:16:46.614472 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.606573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:35075 (local address 127.2.119.193:43679)
0619 08:16:46.610578 (+ 4005us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.610580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.610591 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:16:46.610620 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.610621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.610631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.610687 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:46.610964 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.611700 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.613222 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.613822 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.613881 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.613892 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.614348 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.614394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3971}
I20260619 08:16:46.615059 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.606346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43679 (local address 127.2.119.195:35075)
0619 08:16:46.606653 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.606663 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.606711 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.610751 (+ 4040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:46.610752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:46.610755 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:46.610859 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.610863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.611833 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.611833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.613111 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.613115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.614143 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.614144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.614247 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.614253 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:16:46.614900 (+ 647us) client_negotiation.cc:240] Negotiation successful
0619 08:16:46.614951 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":69,"threads_started":1}
I20260619 08:16:46.615653 15437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd25f674868a4b248fc205def9d0a71c" candidate_uuid: "c361d8f61133407581a313ee2506921c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" is_pre_election: true
I20260619 08:16:46.615849 15437 raft_consensus.cc:2454] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c361d8f61133407581a313ee2506921c in term 0.
I20260619 08:16:46.616104 15641 leader_election.cc:302] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ecb70bacb24b3cb2a5f79466f4a4bd, c361d8f61133407581a313ee2506921c; no voters:
I20260619 08:16:46.616235 15793 raft_consensus.cc:2781] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:16:46.616273 15793 raft_consensus.cc:482] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:16:46.616293 15793 raft_consensus.cc:3037] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:46.617174 15793 raft_consensus.cc:504] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.617336 15793 leader_election.cc:288] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [CANDIDATE]: Term 1 election: Requested vote from peers 72ecb70bacb24b3cb2a5f79466f4a4bd (127.2.119.193:43679), 688d643ca9a242e590382a1bf2049fc3 (127.2.119.194:37029)
I20260619 08:16:46.617539 15437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd25f674868a4b248fc205def9d0a71c" candidate_uuid: "c361d8f61133407581a313ee2506921c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd"
I20260619 08:16:46.617614 15437 raft_consensus.cc:3037] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:46.618461 15437 raft_consensus.cc:2454] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c361d8f61133407581a313ee2506921c in term 1.
I20260619 08:16:46.618636 15641 leader_election.cc:302] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ecb70bacb24b3cb2a5f79466f4a4bd, c361d8f61133407581a313ee2506921c; no voters:
I20260619 08:16:46.618706 15793 raft_consensus.cc:2781] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:46.618836 15793 raft_consensus.cc:686] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 1 LEADER]: Becoming Leader. State: Replica: c361d8f61133407581a313ee2506921c, State: Running, Role: LEADER
I20260619 08:16:46.618918 15793 consensus_queue.cc:237] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } }
I20260619 08:16:46.619468 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.606049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:41137 (local address 127.2.119.194:37029)
0619 08:16:46.606094 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.606097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.606587 (+ 490us) server_negotiation.cc:407] Connection header received
0619 08:16:46.615644 (+ 9057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.615646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.615658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.615688 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:46.616057 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.616742 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.618431 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.618966 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.619022 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.619031 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.619355 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.619401 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:46.619812 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.605829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:37029 (local address 127.2.119.195:41137)
0619 08:16:46.606506 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.606519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.615547 (+ 9028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.615749 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:46.615750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:46.615751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:46.615962 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.615965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.616946 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.616947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.618327 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:46.618330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.619177 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.619178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:46.619255 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.619259 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:46.619682 (+ 423us) client_negotiation.cc:240] Negotiation successful
0619 08:16:46.619728 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":109,"threads_started":1}
I20260619 08:16:46.619733 15284 catalog_manager.cc:5495] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c reported cstate change: term changed from 0 to 1, leader changed from <none> to c361d8f61133407581a313ee2506921c (127.2.119.195). New cstate: current_term: 1 leader_uuid: "c361d8f61133407581a313ee2506921c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c361d8f61133407581a313ee2506921c" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37379 } health_report { overall_health: HEALTHY } } }
I20260619 08:16:46.620401 15571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd25f674868a4b248fc205def9d0a71c" candidate_uuid: "c361d8f61133407581a313ee2506921c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688d643ca9a242e590382a1bf2049fc3" is_pre_election: true
I20260619 08:16:46.620558 15571 raft_consensus.cc:2454] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c361d8f61133407581a313ee2506921c in term 0.
I20260619 08:16:46.620674 15570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd25f674868a4b248fc205def9d0a71c" candidate_uuid: "c361d8f61133407581a313ee2506921c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688d643ca9a242e590382a1bf2049fc3"
I20260619 08:16:46.620736 15570 raft_consensus.cc:3037] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:46.623008 15570 raft_consensus.cc:2454] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c361d8f61133407581a313ee2506921c in term 1.
I20260619 08:16:46.645991 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.642581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.2.119.195:37379)
0619 08:16:46.642624 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:46.642626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:46.642633 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:16:46.642673 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:46.642674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:46.642694 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:46.642757 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:16:46.642939 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.643673 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.644798 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:46.645096 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:46.645164 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:46.645446 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:16:46.645587 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:16:46.645663 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:46.645887 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 08:16:46.645910 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:16:46.650543 15570 raft_consensus.cc:1261] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 1 FOLLOWER]: Refusing update from remote peer c361d8f61133407581a313ee2506921c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:46.650935 15798 consensus_queue.cc:1035] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [LEADER]: Connected to new peer: Peer: permanent_uuid: "688d643ca9a242e590382a1bf2049fc3" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:46.653651 15437 raft_consensus.cc:1261] T fd25f674868a4b248fc205def9d0a71c P 72ecb70bacb24b3cb2a5f79466f4a4bd [term 1 FOLLOWER]: Refusing update from remote peer c361d8f61133407581a313ee2506921c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:46.653949 15798 consensus_queue.cc:1035] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ecb70bacb24b3cb2a5f79466f4a4bd" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:46.658444 15801 mvcc.cc:205] Tried to move back new op lower bound from 7298486299237064704 to 7298486299111587840. Current Snapshot: MvccSnapshot[applied={T|T < 7298486299237064704}]
I20260619 08:16:46.659065 15804 mvcc.cc:205] Tried to move back new op lower bound from 7298486299237064704 to 7298486299111587840. Current Snapshot: MvccSnapshot[applied={T|T < 7298486299237064704}]
I20260619 08:16:46.661432 15284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:46.661554 15284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:46.663385 15284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e19df7b2e7e74ee0b42bb2a2436576f2: Sending DeleteTablet for 3 replicas of tablet fd25f674868a4b248fc205def9d0a71c
I20260619 08:16:46.663743 15417 tablet_service.cc:1514] Processing DeleteTablet for tablet fd25f674868a4b248fc205def9d0a71c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:46 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:37514
I20260619 08:16:46.664100 15551 tablet_service.cc:1514] Processing DeleteTablet for tablet fd25f674868a4b248fc205def9d0a71c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:46 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:39914
I20260619 08:16:46.664348 15685 tablet_service.cc:1514] Processing DeleteTablet for tablet fd25f674868a4b248fc205def9d0a71c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:46 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:40552
I20260619 08:16:46.664783 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15322
I20260619 08:16:46.665874 15802 mvcc.cc:205] Tried to move back new op lower bound from 7298486299237064704 to 7298486299111587840. Current Snapshot: MvccSnapshot[applied={T|T < 7298486299237064704}]
I20260619 08:16:46.669410 15810 tablet_replica.cc:330] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3: stopping tablet replica
I20260619 08:16:46.669589 15811 tablet_replica.cc:330] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: stopping tablet replica
I20260619 08:16:46.669714 15811 raft_consensus.cc:2229] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:16:46.669848 15811 raft_consensus.cc:2258] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:16:46.673544 15271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:43679 (error 108)
I20260619 08:16:46.673707 15810 raft_consensus.cc:2229] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:46.673913 15810 raft_consensus.cc:2258] T fd25f674868a4b248fc205def9d0a71c P 688d643ca9a242e590382a1bf2049fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:46.674435 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.674264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43679 (local address 127.0.0.1:37518)
0619 08:16:46.674367 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.674400 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260619 08:16:46.674573 15271 catalog_manager.cc:4584] TS 72ecb70bacb24b3cb2a5f79466f4a4bd (127.2.119.193:43679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd25f674868a4b248fc205def9d0a71c: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43679: connect: Connection refused (error 111)
I20260619 08:16:46.674760 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15487
I20260619 08:16:46.675415 15811 ts_tablet_manager.cc:1905] T fd25f674868a4b248fc205def9d0a71c P c361d8f61133407581a313ee2506921c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:46.681977 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15620
I20260619 08:16:46.683158 15779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:46.683050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:37029 (local address 127.0.0.1:39918)
0619 08:16:46.683099 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.683123 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:37029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:16:46.694010 15779 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:16:46.693686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37379 (local address 127.0.0.1:40574)
0619 08:16:46.693737 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:46.693748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:46.693791 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:46.693945 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.195:37379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 08:16:46.697031 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15251
I20260619 08:16:46.706719 2527 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 08:16:47.051254 2527 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 08:16:47.051348 2527 mini_ranger.cc:66] Stopping Ranger...
I20260619 08:16:47.395949 2527 mini_ranger.cc:68] Stopped Ranger
2026-06-19 08:16:47.396 UTC [14317] LOG: received smart shutdown request
2026-06-19 08:16:47.400 UTC [14317] LOG: background worker "logical replication launcher" (PID 14326) exited with exit code 1
2026-06-19 08:16:47.400 UTC [14321] LOG: shutting down
2026-06-19 08:16:47.591 UTC [14317] LOG: database system is shut down
2026-06-19T08:16:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48811 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:16:47 dist-test-slave-d0b5 krb5kdc[15822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:16:47 dist-test-slave-d0b5 krb5kdc[15822](info): set up 2 sockets
Jun 19 08:16:47 dist-test-slave-d0b5 krb5kdc[15822](info): commencing operation
krb5kdc: starting...
W20260619 08:16:48.766242 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:16:48 dist-test-slave-d0b5 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:16:48.896667 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:33191
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:48.984815 15835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:48.985059 15835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:48.985098 15835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:48.986358 15835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:48.986424 15835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:48.986449 15835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:48.986469 15835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:48.986501 15835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:48.988649 15835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:33191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:48.988922 15835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:48.989185 15835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:48.994243 15843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:48.994261 15841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:48.994256 15840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:48.994401 15835 server_base.cc:1034] running on GCE node
I20260619 08:16:48.994686 15835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:48.994925 15835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:48.996083 15835 hybrid_clock.cc:648] HybridClock initialized: now 1781857008996068 us; error 26 us; skew 500 ppm
Jun 19 08:16:48 dist-test-slave-d0b5 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:48.998703 15835 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:48.999185 15835 webserver.cc:460] Webserver started at http://127.2.119.254:33527/ using document root <none> and password file <none>
I20260619 08:16:48.999469 15835 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:48.999543 15835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:48.999691 15835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:49.000728 15835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "ea0cb30e739d4f85ab012e0d6d07ce59"
format_stamp: "Formatted at 2026-06-19 08:16:48 on dist-test-slave-d0b5"
server_key: "20befc03eef6e99b010308d4ccc37648"
server_key_iv: "76bd470c624af5bc11cb6323674b9ad4"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.001093 15835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "ea0cb30e739d4f85ab012e0d6d07ce59"
format_stamp: "Formatted at 2026-06-19 08:16:48 on dist-test-slave-d0b5"
server_key: "20befc03eef6e99b010308d4ccc37648"
server_key_iv: "76bd470c624af5bc11cb6323674b9ad4"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.002524 15835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:49.003286 15850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:49.003509 15835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:49.003609 15835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ea0cb30e739d4f85ab012e0d6d07ce59"
format_stamp: "Formatted at 2026-06-19 08:16:48 on dist-test-slave-d0b5"
server_key: "20befc03eef6e99b010308d4ccc37648"
server_key_iv: "76bd470c624af5bc11cb6323674b9ad4"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.003691 15835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:49.010227 15835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:49.012423 15835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:49.012583 15835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:49.016721 15835 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:33191
I20260619 08:16:49.016758 15902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:33191 every 8 connection(s)
I20260619 08:16:49.017093 15835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:49.018011 15903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:49.019610 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.021550 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15835
I20260619 08:16:49.021660 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:16:49.021855 2527 external_mini_cluster.cc:1428] Setting key 0a94d629c4dcc3b12b2922fee6e95c62
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.253@KRBTEST.COM; defaulting to no policy
I20260619 08:16:49.030153 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.020390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:33191 (local address 127.0.0.1:36064)
0619 08:16:49.020573 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.020612 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.021066 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.021481 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.021484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.021890 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.022137 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.022142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.023499 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.023500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.024217 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.024218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.024754 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.024755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.024820 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.025435 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.025443 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.026806 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.028609 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.028616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.028618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.028806 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.029054 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.029055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.029056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.029137 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.029570 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.029576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.029647 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.029746 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.029982 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":98,"thread_start_us":77,"threads_started":1}
I20260619 08:16:49.030153 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.020555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36064 (local address 127.2.119.254:33191)
0619 08:16:49.020678 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.020683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.020721 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:16:49.021236 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.021238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.021264 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.021440 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.022267 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.023384 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.024332 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.024652 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.024799 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.026906 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.026913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.026913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.026920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.028506 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.028901 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.028903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.028904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.028984 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.029205 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.029206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.029206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.029495 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.029591 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.029779 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.029904 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260619 08:16:49.031111 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.031505 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.031381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45270)
0619 08:16:49.031433 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.031483 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:16:49.031635 15852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.253@KRBTEST.COM" created.
I20260619 08:16:49.032114 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.032017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45284)
0619 08:16:49.032062 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.032087 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:16:49.032200 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:34931: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
I20260619 08:16:49.049805 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:34931
--webserver_interface=127.2.119.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:49.081411 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } attempt: 1
I20260619 08:16:49.081995 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.081843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45290)
0619 08:16:49.081909 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.081958 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:16:49.082446 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.082303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45294)
0619 08:16:49.082389 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.082421 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 08:16:49.082602 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:34931: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
W20260619 08:16:49.132128 15910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:49.132382 15910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:49.132418 15910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:49.133608 15910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:49.133678 15910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:49.133702 15910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:49.133724 15910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:49.133766 15910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:49.135318 15910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:34931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.119.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:49.135599 15910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:49.135845 15910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:49.140717 15918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:49.140738 15915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:49.140841 15910 server_base.cc:1034] running on GCE node
W20260619 08:16:49.141024 15916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:49.141276 15910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:49.141609 15910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:49.142802 15910 hybrid_clock.cc:648] HybridClock initialized: now 1781857009142795 us; error 43 us; skew 500 ppm
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:49.145447 15910 init.cc:377] Logged in from keytab as kudu/127.2.119.253@KRBTEST.COM (short username kudu)
I20260619 08:16:49.145923 15910 webserver.cc:460] Webserver started at http://127.2.119.253:42999/ using document root <none> and password file <none>
I20260619 08:16:49.146188 15910 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:49.146260 15910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:49.146415 15910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:49.147527 15910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data/instance:
uuid: "29cc9de982b64a158cbed9028e24b166"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "233b679c2263ee5dcac2d934b6d53db1"
server_key_iv: "749d20168e94ca0fe1f093c5a748bf7b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.147894 15910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal/instance:
uuid: "29cc9de982b64a158cbed9028e24b166"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "233b679c2263ee5dcac2d934b6d53db1"
server_key_iv: "749d20168e94ca0fe1f093c5a748bf7b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.149240 15910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:16:49.150090 15925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:49.150334 15910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:49.150494 15910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "29cc9de982b64a158cbed9028e24b166"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "233b679c2263ee5dcac2d934b6d53db1"
server_key_iv: "749d20168e94ca0fe1f093c5a748bf7b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.150594 15910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:49.150777 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } attempt: 2
I20260619 08:16:49.151228 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.151086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45296)
0619 08:16:49.151163 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.151199 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 08:16:49.151659 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.151511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45304)
0619 08:16:49.151571 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.151619 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 08:16:49.151806 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:34931: Network error: Client connection negotiation failed: client connection to 127.2.119.253:34931: connect: Connection refused (error 111)
I20260619 08:16:49.178292 15910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:49.180781 15910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:49.180939 15910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:49.184976 15910 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.253:34931
I20260619 08:16:49.185040 15977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.253:34931 every 8 connection(s)
I20260619 08:16:49.185527 15910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
I20260619 08:16:49.186250 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:49.187814 15978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:49.194960 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15910
I20260619 08:16:49.195066 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-1/wal/instance
I20260619 08:16:49.195245 2527 external_mini_cluster.cc:1428] Setting key 09114db60849c477e0e8f31e9cff179b
I20260619 08:16:49.197237 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.188558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.2.119.254:33191)
0619 08:16:49.188644 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.188646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.188817 (+ 171us) server_negotiation.cc:407] Connection header received
0619 08:16:49.189280 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.189282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.189366 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.189442 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.190068 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.190594 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.191657 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.191865 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.191975 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.194055 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.194057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.194057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.194063 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.195625 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.196118 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.196119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.196119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.196172 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.196500 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.196501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.196501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.196679 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.196754 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.197031 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.197122 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 08:16:49.197258 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.188529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:33191 (local address 127.0.0.1:36076)
0619 08:16:49.188683 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.188726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.189160 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.189546 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.189549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.189788 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.189978 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.189982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.190677 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.190678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.191489 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.191491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.191984 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.191985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.192093 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.192568 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.192577 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.193963 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.195704 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.195710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.195713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.196041 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.196242 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.196243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.196243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.196384 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.196765 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.196771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.196889 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.196986 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.197104 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260619 08:16:49.198077 15978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.252@KRBTEST.COM" created.
I20260619 08:16:49.206674 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.198360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45320)
0619 08:16:49.198417 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.198426 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.198467 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.198739 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.198740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.198874 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.198919 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.198920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.200362 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.200363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.200858 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.200860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.201572 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.201573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.201632 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.202093 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.202097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.203434 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.205223 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.205224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.205224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.205522 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.205740 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.205741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.205741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.205825 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.206313 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.206314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.206361 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.206431 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.206535 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:16:49.206799 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.198433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.2.119.253:34931)
0619 08:16:49.198600 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.198604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.198620 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:16:49.198636 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.198637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.198648 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.198725 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.199029 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.200213 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.200940 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.201475 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.201568 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.203534 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.203543 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.203544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.203551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.205130 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.205615 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.205618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.205618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.205672 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.206008 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.206009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.206009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.206219 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.206313 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.206505 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.206607 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260619 08:16:49.207557 15978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.207926 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.207807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60694)
0619 08:16:49.207858 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.207907 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:16:49.208108 15929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
I20260619 08:16:49.208559 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.208442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60696)
0619 08:16:49.208510 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.208536 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 08:16:49.208657 15978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:41149: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
I20260619 08:16:49.220781 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:41149
--webserver_interface=127.2.119.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:49.242959 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } attempt: 3
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:16:49.250854 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.243388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45334)
0619 08:16:49.243457 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.243469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.243546 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.243694 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.243695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.243955 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.244028 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.244030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.244737 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.244740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.245342 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.245345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.245735 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.245736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.245827 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.246314 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.246326 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.247756 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.249722 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.249723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.249724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.249918 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.250154 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.250155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.250155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.250224 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.250492 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.250494 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.250544 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.250634 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.250746 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:16:49.250862 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.243476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.2.119.253:34931)
0619 08:16:49.243549 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.243551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.243572 (+ 21us) server_negotiation.cc:407] Connection header received
0619 08:16:49.243605 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.243606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.243626 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.243691 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.244119 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.244635 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.245433 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.245655 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.245739 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.247868 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.247869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.247870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.247878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.249581 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.250010 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.250012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.250013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.250081 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.250292 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.250292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.250293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.250421 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.250502 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.250659 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.250748 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:16:49.251238 15903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.251612 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.251506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60702)
0619 08:16:49.251559 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.251587 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 08:16:49.252030 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.251945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60704)
0619 08:16:49.251990 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.252014 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:16:49.252135 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:41149: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
I20260619 08:16:49.257895 15978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } attempt: 1
I20260619 08:16:49.258287 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.258171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60708)
0619 08:16:49.258228 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.258262 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:16:49.258634 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.258538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60710)
0619 08:16:49.258585 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.258608 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:16:49.258718 15978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:41149: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
I20260619 08:16:49.283329 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } attempt: 1
I20260619 08:16:49.283807 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.283667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60724)
0619 08:16:49.283733 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.283773 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:16:49.284166 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.284070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60728)
0619 08:16:49.284116 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.284143 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 08:16:49.284262 15903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:41149: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
W20260619 08:16:49.303634 15985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:49.303864 15985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:49.303900 15985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:49.305176 15985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:49.305245 15985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:49.305270 15985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:49.305325 15985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:49.305356 15985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:49.307024 15985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:33191,127.2.119.253:34931,127.2.119.252:41149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.119.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:49.307299 15985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:49.307570 15985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:49.312554 15990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:49.312662 15985 server_base.cc:1034] running on GCE node
W20260619 08:16:49.312575 15993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:49.312554 15991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:49.313059 15985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:49.313355 15985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:49.314520 15985 hybrid_clock.cc:648] HybridClock initialized: now 1781857009314508 us; error 29 us; skew 500 ppm
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:49.317289 15985 init.cc:377] Logged in from keytab as kudu/127.2.119.252@KRBTEST.COM (short username kudu)
I20260619 08:16:49.317806 15985 webserver.cc:460] Webserver started at http://127.2.119.252:44581/ using document root <none> and password file <none>
I20260619 08:16:49.318056 15985 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:49.318101 15985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:49.318243 15985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:49.319389 15985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data/instance:
uuid: "08a57115211d4b40a1bcb4d5dcdee9d8"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "32a656d32608432c2c2d9d38798d7e33"
server_key_iv: "1be6eefa3f32be3f72ee85e7bfe8389b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.319761 15985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal/instance:
uuid: "08a57115211d4b40a1bcb4d5dcdee9d8"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "32a656d32608432c2c2d9d38798d7e33"
server_key_iv: "1be6eefa3f32be3f72ee85e7bfe8389b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.321107 15985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:16:49.321907 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:49.322099 15985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:49.322163 15985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "08a57115211d4b40a1bcb4d5dcdee9d8"
format_stamp: "Formatted at 2026-06-19 08:16:49 on dist-test-slave-d0b5"
server_key: "32a656d32608432c2c2d9d38798d7e33"
server_key_iv: "1be6eefa3f32be3f72ee85e7bfe8389b"
server_key_version: "encryptionkey@0"
I20260619 08:16:49.322256 15985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:49.326920 15978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } attempt: 2
I20260619 08:16:49.327324 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.327200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60732)
0619 08:16:49.327259 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.327291 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:16:49.327726 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.327642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60742)
0619 08:16:49.327685 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.327710 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 08:16:49.327836 15978 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:41149: Network error: Client connection negotiation failed: client connection to 127.2.119.252:41149: connect: Connection refused (error 111)
I20260619 08:16:49.340590 15985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:49.342660 15985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:49.342814 15985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:49.346938 15985 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.252:41149
I20260619 08:16:49.346966 16052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.252:41149 every 8 connection(s)
I20260619 08:16:49.347313 15985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
I20260619 08:16:49.348209 16053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:49.349871 16053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.355602 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 15985
I20260619 08:16:49.355692 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781856856097588-2527-0/minicluster-data/master-2/wal/instance
I20260619 08:16:49.355899 2527 external_mini_cluster.cc:1428] Setting key 188c7cf90c2269060607b71253a75419
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:49.359426 15903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } attempt: 2
I20260619 08:16:49.359905 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.350637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36090 (local address 127.2.119.254:33191)
0619 08:16:49.350690 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.350692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.350823 (+ 131us) server_negotiation.cc:407] Connection header received
0619 08:16:49.351271 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.351272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.351282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.351354 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.352054 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.352543 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.353852 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.354112 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.354175 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.356798 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.356799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.356800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.356809 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.358344 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.358764 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.358765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.358766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.358829 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.359109 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.359109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.359110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.359254 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.359305 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.359701 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.359796 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:49.360375 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.350534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:33191 (local address 127.0.0.1:36090)
0619 08:16:49.350693 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.350734 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.351177 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.351446 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.351450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.351708 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.351953 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.351958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.352655 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.352656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.353710 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.353713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.354201 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.354202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.354321 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.354978 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.354987 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.356709 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.358450 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.358455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.358457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.358684 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.358924 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.358925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.358925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.359035 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.359374 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.359379 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.359534 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.359666 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.359832 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260619 08:16:49.360940 16053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:16:49.366094 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.356289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.2.119.252:41149)
0619 08:16:49.356498 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.356504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.356524 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:16:49.356548 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.356550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.356572 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.356665 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.357405 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.358307 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.359204 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.359663 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.359722 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.362489 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.362497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.362498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.362504 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.364710 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.365151 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.365154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.365154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.365235 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.365508 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.365509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.365509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.365653 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.365804 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.365881 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.365966 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260619 08:16:49.368516 15905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.359681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60760)
0619 08:16:49.359764 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.359773 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.359810 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.360250 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.360251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.360417 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.360459 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.360461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.361399 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.361400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.362179 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.362182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.362670 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.362671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.362729 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.363147 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.363151 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.364581 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.367460 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.367461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.367694 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.367883 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.367885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.367949 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.368201 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.368202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.368247 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.368327 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.368419 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 08:16:49.368543 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.361188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:34931 (local address 127.0.0.1:45342)
0619 08:16:49.361225 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.361234 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.361268 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.361439 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.361440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.361572 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.361616 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.361617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.362539 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.362539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.363295 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.363298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.363747 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.363748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.363801 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.364190 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.364195 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.365781 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.367560 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.367561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.367728 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.367937 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.367938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.367989 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.368292 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.368293 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.368324 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.368379 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.368457 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:16:49.368621 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.361257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.2.119.253:34931)
0619 08:16:49.361333 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.361335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.361343 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:16:49.361368 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.361369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.361381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.361434 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.361823 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.362452 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.363381 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.363638 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.363731 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.365893 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.365894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.365894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.365903 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.367497 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367811 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.367812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.367812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.367871 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.368074 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.368075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.368075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.368225 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.368302 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.368429 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.368501 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:16:49.368902 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.359887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.2.119.252:41149)
0619 08:16:49.360088 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.360090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.360097 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:16:49.360115 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.360115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.360124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.360232 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.360556 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.361320 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.362263 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.362578 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.362870 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.364672 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.364673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.364673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.364681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.367382 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.367773 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.367774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.367774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.367830 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.368005 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.368006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.368006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.368128 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.368196 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.368713 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.368794 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":144,"threads_started":1}
I20260619 08:16:49.368912 16053 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:16:49.371068 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: Bootstrap starting.
I20260619 08:16:49.371852 15903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:49.372203 15903 log.cc:824] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: Log is configured to *not* fsync() on all Append() calls
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:49.373049 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: No bootstrap required, opened a new log
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:16:49.375808 15903 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.376186 15903 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:49.376325 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.367184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.2.119.254:33191)
0619 08:16:49.367225 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.367226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.367234 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:16:49.367286 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.367287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.367296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.367350 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.367932 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.368626 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.369827 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.370108 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.370179 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.373373 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.373375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.373375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.373382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.375016 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.375445 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.375446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.375446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.375508 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.375709 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.375711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.375711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.375882 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.376029 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.376100 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.376184 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:49.376310 15903 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0cb30e739d4f85ab012e0d6d07ce59, State: Initialized, Role: FOLLOWER
I20260619 08:16:49.376514 15903 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.376657 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.369254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:60766)
0619 08:16:49.369348 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.369357 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.369396 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.369558 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.369558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.369687 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.369710 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.369711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.370537 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.370538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.371205 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.371207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.371620 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.371620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.371671 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.372073 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.372077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.373957 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.375554 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.375555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.375555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.375767 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.375996 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.375997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.375997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.376050 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.376360 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.376361 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.376402 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.376478 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.376564 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 08:16:49.376906 15903 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:49.376953 16060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.377012 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.369325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60766 (local address 127.2.119.252:41149)
0619 08:16:49.369478 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.369479 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.369485 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:16:49.369501 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.369502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.369511 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.369556 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.369793 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.370449 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.371287 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.371523 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.371618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.374057 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.374058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.374058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.374065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.375451 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.375863 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.375864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.375865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.375921 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.376106 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.376107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.376107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.376260 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.376326 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.376838 (+ 512us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.376912 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260619 08:16:49.377082 16060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.378736 16053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: Bootstrap starting.
W20260619 08:16:49.378944 16071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:16:49.378991 16071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:16:49.379473 16053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:49.379796 16053 log.cc:824] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:49.382048 16053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: No bootstrap required, opened a new log
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:16:49.379729 15835 master_runner.cc:186] Error getting master registration for 127.2.119.253:34931: OK, instance_id { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" instance_seqno: 1781857009183886 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:16:49.382768 15835 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:16:49.383997 16053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.384104 16053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:49.384124 16053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a57115211d4b40a1bcb4d5dcdee9d8, State: Initialized, Role: FOLLOWER
I20260619 08:16:49.384246 16053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.384490 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.384537 16053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:49.384562 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.386413 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.376874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.2.119.253:34931)
0619 08:16:49.376918 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.376920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.377024 (+ 104us) server_negotiation.cc:407] Connection header received
0619 08:16:49.377279 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.377281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.377320 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.377386 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.378073 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.378764 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.379692 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.379984 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.380043 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.382856 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.382858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.382858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.382867 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.384574 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.384961 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.384962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.384962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.385022 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.385212 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.385213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.385213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.385396 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.386025 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.386225 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.386309 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 08:16:49.386744 16085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:16:49.386838 16085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:16:49.387754 15985 master_runner.cc:186] Error getting master registration for 127.2.119.253:34931: OK, instance_id { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" instance_seqno: 1781857009183886 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:16:49.389253 15985 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:16:49.389362 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:16:49.397588 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.390745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.2.119.254:33191)
0619 08:16:49.390788 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.390790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.391040 (+ 250us) server_negotiation.cc:407] Connection header received
0619 08:16:49.391111 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.391112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.391122 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.391177 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.391716 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.392186 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.392888 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.393098 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.393175 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.394629 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.394631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.394631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.394639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.396296 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.396716 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.396717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.396718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.396778 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.396998 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.396999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.396999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.397132 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.397238 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.397398 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.397473 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:16:49.404891 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.390809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.2.119.253:34931)
0619 08:16:49.390844 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.390846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.397604 (+ 6758us) server_negotiation.cc:407] Connection header received
0619 08:16:49.397703 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.397704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.397714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.397744 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.398366 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.399107 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.400083 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.400395 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.400469 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.402012 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.402013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.402014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.402020 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.403615 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.404032 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.404033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.404033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.404091 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.404286 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.404287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.404287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.404404 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.404500 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.404716 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.404795 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:16:49.409291 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.390840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.2.119.252:41149)
0619 08:16:49.390877 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.390878 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.401443 (+ 10565us) server_negotiation.cc:407] Connection header received
0619 08:16:49.401554 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.401555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.401567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.401598 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.402568 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.403572 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.404573 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.404868 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.404924 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.406457 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.406459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.406459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.406466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.408060 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.408450 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.408451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.408451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.408509 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.408715 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.408716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.408716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.408848 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.408942 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.409116 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.409193 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:16:49.418970 15978 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } attempt: 3
Jun 19 08:16:49 dist-test-slave-d0b5 krb5kdc[15822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857009, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:16:49.427479 15980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.419215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:41149 (local address 127.0.0.1:49964)
0619 08:16:49.419261 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:49.419271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:49.419304 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:49.419427 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:49.419428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:49.419600 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:49.419648 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.419650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.420563 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.420564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.421453 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:49.421455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.421917 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.421918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:49.421977 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.422458 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:49.422463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:49.423988 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:49.425914 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.425915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.425915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.426133 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.426464 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:49.426465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:49.426466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:49.426521 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:49.426788 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:49.426789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:49.427093 (+ 304us) client_negotiation.cc:769] Sending connection context
0619 08:16:49.427175 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:16:49.427328 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 08:16:49.428072 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:49.419261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.2.119.252:41149)
0619 08:16:49.419299 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:49.419301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:49.419317 (+ 16us) server_negotiation.cc:407] Connection header received
0619 08:16:49.419357 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:49.419358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:49.419367 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:49.419427 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:49.419723 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.420476 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.421535 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:49.421835 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:49.421909 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:49.424101 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:49.424102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:49.424102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:49.424109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:49.425815 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.426337 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.426338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.426338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.426393 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:49.426595 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:49.426596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:49.426596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:49.426717 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:49.427770 (+ 1053us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:49.427837 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 08:16:49.427938 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:16:49.430130 15978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: Bootstrap starting.
I20260619 08:16:49.430886 15978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:49.431180 15978 log.cc:824] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:49.431988 15978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: No bootstrap required, opened a new log
I20260619 08:16:49.434021 15978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.434170 15978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:49.434206 15978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29cc9de982b64a158cbed9028e24b166, State: Initialized, Role: FOLLOWER
I20260619 08:16:49.434331 15978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.434643 15978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:49.434617 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.434705 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 08:16:49.436429 16106 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:16:49.436544 16106 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:16:49.438881 15910 master_runner.cc:263] No leader master found from master 29cc9de982b64a158cbed9028e24b166
I20260619 08:16:49.438932 15910 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:16:49.488524 16060 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:16:49.488646 16060 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.488897 16060 leader_election.cc:288] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29cc9de982b64a158cbed9028e24b166 (127.2.119.253:34931), 08a57115211d4b40a1bcb4d5dcdee9d8 (127.2.119.252:41149)
I20260619 08:16:49.489501 15952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cc9de982b64a158cbed9028e24b166" is_pre_election: true
I20260619 08:16:49.489666 15952 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea0cb30e739d4f85ab012e0d6d07ce59 in term 0.
I20260619 08:16:49.490147 15852 leader_election.cc:302] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cc9de982b64a158cbed9028e24b166, ea0cb30e739d4f85ab012e0d6d07ce59; no voters:
I20260619 08:16:49.490346 16060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:16:49.490406 16060 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:16:49.490429 16060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:49.491375 16060 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.491506 16060 leader_election.cc:288] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [CANDIDATE]: Term 1 election: Requested vote from peers 29cc9de982b64a158cbed9028e24b166 (127.2.119.253:34931), 08a57115211d4b40a1bcb4d5dcdee9d8 (127.2.119.252:41149)
I20260619 08:16:49.491794 15952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29cc9de982b64a158cbed9028e24b166"
I20260619 08:16:49.491866 15952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:49.492766 15952 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea0cb30e739d4f85ab012e0d6d07ce59 in term 1.
I20260619 08:16:49.492987 15852 leader_election.cc:302] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29cc9de982b64a158cbed9028e24b166, ea0cb30e739d4f85ab012e0d6d07ce59; no voters:
I20260619 08:16:49.493242 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" is_pre_election: true
I20260619 08:16:49.493425 16027 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea0cb30e739d4f85ab012e0d6d07ce59 in term 0.
I20260619 08:16:49.493789 16027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8"
I20260619 08:16:49.493858 16027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:49.494679 16027 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea0cb30e739d4f85ab012e0d6d07ce59 in term 1.
I20260619 08:16:49.495875 16060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:49.496030 16060 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [term 1 LEADER]: Becoming Leader. State: Replica: ea0cb30e739d4f85ab012e0d6d07ce59, State: Running, Role: LEADER
I20260619 08:16:49.496117 16060 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } }
I20260619 08:16:49.496515 16060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea0cb30e739d4f85ab012e0d6d07ce59. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.496584 16060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:49.497952 16110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:49.498205 16110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:49.502283 16027 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [term 1 FOLLOWER]: Refusing update from remote peer ea0cb30e739d4f85ab012e0d6d07ce59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:49.502688 16060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:49.504173 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea0cb30e739d4f85ab012e0d6d07ce59. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.504278 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.504750 15952 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [term 1 FOLLOWER]: Refusing update from remote peer ea0cb30e739d4f85ab012e0d6d07ce59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:49.505054 16060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:49.506417 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea0cb30e739d4f85ab012e0d6d07ce59. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.506518 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.508899 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.509603 16074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.509684 16074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.509902 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:49.510097 16111 mvcc.cc:205] Tried to move back new op lower bound from 7298486310918619136 to 7298486310896570368. Current Snapshot: MvccSnapshot[applied={T|T < 7298486310918619136}]
I20260619 08:16:49.510231 16112 mvcc.cc:205] Tried to move back new op lower bound from 7298486310918619136 to 7298486310896570368. Current Snapshot: MvccSnapshot[applied={T|T < 7298486310918619136}]
I20260619 08:16:49.511457 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.511541 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:49.512024 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cb30e739d4f85ab012e0d6d07ce59" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 33191 } } peers { permanent_uuid: "29cc9de982b64a158cbed9028e24b166" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 34931 } } peers { permanent_uuid: "08a57115211d4b40a1bcb4d5dcdee9d8" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 41149 } } }
I20260619 08:16:49.512105 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:16:49.512408 16114 mvcc.cc:205] Tried to move back new op lower bound from 7298486310918619136 to 7298486310896570368. Current Snapshot: MvccSnapshot[applied={T|T < 7298486310918619136}]
I20260619 08:16:49.513752 16110 catalog_manager.cc:1347] Generated new cluster ID: c796a81b7a6440ff842614c887a50ba5
I20260619 08:16:49.513804 16110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:16:49.553053 16110 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:16:49.555030 16110 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:49.567018 16110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea0cb30e739d4f85ab012e0d6d07ce59: Generated new TSK 0
I20260619 08:16:49.567327 16110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:16:50.387271 16085 catalog_manager.cc:1259] Loaded cluster ID: c796a81b7a6440ff842614c887a50ba5
I20260619 08:16:50.387387 16085 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: loading cluster ID for follower catalog manager: success
I20260619 08:16:50.388235 16085 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: acquiring CA information for follower catalog manager: success
I20260619 08:16:50.388551 16085 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 08a57115211d4b40a1bcb4d5dcdee9d8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:16:50.437012 16106 catalog_manager.cc:1259] Loaded cluster ID: c796a81b7a6440ff842614c887a50ba5
I20260619 08:16:50.437109 16106 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: loading cluster ID for follower catalog manager: success
I20260619 08:16:50.437991 16106 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: acquiring CA information for follower catalog manager: success
I20260619 08:16:50.438313 16106 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29cc9de982b64a158cbed9028e24b166: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:16:50.617749 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15835
I20260619 08:16:50.623451 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15910
I20260619 08:16:50.628430 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 15985
2026-06-19T08:16:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2924 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:16:50 dist-test-slave-d0b5 krb5kdc[16125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:16:50 dist-test-slave-d0b5 krb5kdc[16125](info): set up 2 sockets
Jun 19 08:16:50 dist-test-slave-d0b5 krb5kdc[16125](info): commencing operation
krb5kdc: starting...
W20260619 08:16:51.666455 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:16:51 dist-test-slave-d0b5 krb5kdc[16125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:16:51.786298 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36975
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:51.865729 16138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:51.865976 16138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:51.866024 16138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:51.867166 16138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:51.867237 16138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:51.867262 16138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:51.867336 16138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:51.867396 16138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:51.868892 16138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:51.869153 16138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:51.869426 16138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:51.874478 16143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:51.874503 16144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:51.874655 16138 server_base.cc:1034] running on GCE node
W20260619 08:16:51.874477 16146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:51.874953 16138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:51.875200 16138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:51.876344 16138 hybrid_clock.cc:648] HybridClock initialized: now 1781857011876321 us; error 50 us; skew 500 ppm
Jun 19 08:16:51 dist-test-slave-d0b5 krb5kdc[16125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857011, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:51.878944 16138 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:51.879329 16138 webserver.cc:460] Webserver started at http://127.2.119.254:42553/ using document root <none> and password file <none>
I20260619 08:16:51.879571 16138 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:51.879642 16138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:51.879796 16138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:51.880880 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "55b1d6fa02bb49629e53962218f25c71"
format_stamp: "Formatted at 2026-06-19 08:16:51 on dist-test-slave-d0b5"
server_key: "4d1c85fae0c1a037502e88d67b763e46"
server_key_iv: "4e865cfc141a849ce800aec69f17c5e9"
server_key_version: "encryptionkey@0"
I20260619 08:16:51.881251 16138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "55b1d6fa02bb49629e53962218f25c71"
format_stamp: "Formatted at 2026-06-19 08:16:51 on dist-test-slave-d0b5"
server_key: "4d1c85fae0c1a037502e88d67b763e46"
server_key_iv: "4e865cfc141a849ce800aec69f17c5e9"
server_key_version: "encryptionkey@0"
I20260619 08:16:51.882648 16138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:51.883438 16153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:51.883639 16138 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:51.883747 16138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "55b1d6fa02bb49629e53962218f25c71"
format_stamp: "Formatted at 2026-06-19 08:16:51 on dist-test-slave-d0b5"
server_key: "4d1c85fae0c1a037502e88d67b763e46"
server_key_iv: "4e865cfc141a849ce800aec69f17c5e9"
server_key_version: "encryptionkey@0"
I20260619 08:16:51.883823 16138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:51.901202 16138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:51.903555 16138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:51.903723 16138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:51.907665 16138 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36975
I20260619 08:16:51.908032 16205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36975 every 8 connection(s)
I20260619 08:16:51.908036 16138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:51.908735 16206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:51.910883 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16138
I20260619 08:16:51.911003 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:16:51.911175 2527 external_mini_cluster.cc:1428] Setting key 6736afd0caeb8a1d7a04a2fc515c146c
I20260619 08:16:51.911262 16206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71: Bootstrap starting.
I20260619 08:16:51.911962 16206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:51.912268 16206 log.cc:824] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:51.912974 16206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71: No bootstrap required, opened a new log
I20260619 08:16:51.914839 16206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } }
I20260619 08:16:51.915004 16206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:51.915067 16206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b1d6fa02bb49629e53962218f25c71, State: Initialized, Role: FOLLOWER
I20260619 08:16:51.915221 16206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } }
I20260619 08:16:51.915292 16206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:51.915313 16206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:51.915380 16206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:51.915941 16206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } }
I20260619 08:16:51.916057 16206 leader_election.cc:302] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b1d6fa02bb49629e53962218f25c71; no voters:
I20260619 08:16:51.916244 16206 leader_election.cc:288] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:16:51.916359 16211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:51.916534 16211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [term 1 LEADER]: Becoming Leader. State: Replica: 55b1d6fa02bb49629e53962218f25c71, State: Running, Role: LEADER
I20260619 08:16:51.916554 16206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:51.916666 16211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } }
I20260619 08:16:51.916978 16211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b1d6fa02bb49629e53962218f25c71. Latest consensus state: current_term: 1 leader_uuid: "55b1d6fa02bb49629e53962218f25c71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } } }
I20260619 08:16:51.917191 16211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:51.917120 16212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55b1d6fa02bb49629e53962218f25c71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b1d6fa02bb49629e53962218f25c71" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36975 } } }
I20260619 08:16:51.917640 16212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:51.918052 16220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 19 08:16:51 dist-test-slave-d0b5 krb5kdc[16125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:51.918648 16220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:51.921003 16220 catalog_manager.cc:1347] Generated new cluster ID: 4dfc22b0c417455ba7dc804c3b423421
I20260619 08:16:51.921053 16220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:16:51.922524 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:51.911723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.2.119.254:36975)
0619 08:16:51.911906 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:51.911914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:51.911939 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:16:51.912468 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:51.912476 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:51.912525 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:51.912677 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:51.913397 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:51.914275 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:51.915054 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:51.915637 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:51.915752 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:51.918802 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:51.918806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:51.918806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:51.918818 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:51.920845 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:51.921238 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:51.921241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:51.921242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:51.921372 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:51.921596 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:51.921597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:51.921597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:51.921925 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:51.922090 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:51.922172 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:16:51.922311 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260619 08:16:51.926338 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:16:51.928876 16220 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:16:51.929463 16220 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:51.931926 16220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b1d6fa02bb49629e53962218f25c71: Generated new TSK 0
I20260619 08:16:51.932155 16220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:16:51.980469 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16138
2026-06-19T08:16:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1351 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:16:52 dist-test-slave-d0b5 krb5kdc[16233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:16:52 dist-test-slave-d0b5 krb5kdc[16233](info): set up 2 sockets
Jun 19 08:16:52 dist-test-slave-d0b5 krb5kdc[16233](info): commencing operation
krb5kdc: starting...
W20260619 08:16:53.027138 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:16:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:16:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:16:53.151803 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:53.232806 16246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:53.233065 16246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:53.233105 16246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:53.234340 16246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:53.234417 16246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:53.234441 16246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:53.234472 16246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:53.234501 16246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:53.236025 16246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:53.236318 16246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:53.236562 16246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:53.241953 16254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.241976 16251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.241971 16252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.242010 16246 server_base.cc:1034] running on GCE node
I20260619 08:16:53.242435 16246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:53.242753 16246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:53.243917 16246 hybrid_clock.cc:648] HybridClock initialized: now 1781857013243891 us; error 36 us; skew 500 ppm
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:53.250700 16246 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:53.251201 16246 webserver.cc:460] Webserver started at http://127.2.119.254:40869/ using document root <none> and password file <none>
I20260619 08:16:53.251451 16246 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:53.251503 16246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:53.251646 16246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:53.252723 16246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "ca02f7da742743e5b7d20b4c704e2143"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "abe8b3c8918c8a9d8bcf28ff4c71f2ff"
server_key_iv: "f03b4987ddc438ec98db36328240fea0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.253084 16246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "ca02f7da742743e5b7d20b4c704e2143"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "abe8b3c8918c8a9d8bcf28ff4c71f2ff"
server_key_iv: "f03b4987ddc438ec98db36328240fea0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.254527 16246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:16:53.255290 16261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.255479 16246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:16:53.255590 16246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ca02f7da742743e5b7d20b4c704e2143"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "abe8b3c8918c8a9d8bcf28ff4c71f2ff"
server_key_iv: "f03b4987ddc438ec98db36328240fea0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.255666 16246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:53.264444 16246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:53.266770 16246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:53.266930 16246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:53.271080 16246 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45921
I20260619 08:16:53.271123 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45921 every 8 connection(s)
I20260619 08:16:53.271517 16246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:53.272506 16314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:53.274871 16314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap starting.
I20260619 08:16:53.275389 16314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:53.275633 16314 log.cc:824] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:53.276291 16314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: No bootstrap required, opened a new log
I20260619 08:16:53.276624 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16246
I20260619 08:16:53.276739 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:16:53.276919 2527 external_mini_cluster.cc:1428] Setting key 81c299e2bba6a0b7a1e502d5665bd8d5
I20260619 08:16:53.278308 16314 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:53.278481 16314 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:53.278568 16314 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Initialized, Role: FOLLOWER
I20260619 08:16:53.278738 16314 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:53.278834 16314 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:53.278868 16314 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:53.278928 16314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:53.279924 16314 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:53.280015 16314 leader_election.cc:302] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca02f7da742743e5b7d20b4c704e2143; no voters:
I20260619 08:16:53.280208 16314 leader_election.cc:288] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:16:53.280244 16319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:53.280429 16319 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 LEADER]: Becoming Leader. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Running, Role: LEADER
I20260619 08:16:53.280478 16314 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:53.280601 16319 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:53.281019 16320 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:53.281090 16320 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:53.281348 16321 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca02f7da742743e5b7d20b4c704e2143. Latest consensus state: current_term: 1 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:53.281497 16321 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
W20260619 08:16:53.282626 16334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:16:53.282914 16334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:16:53.282999 16325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:53.283185 16325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:53.284713 16325 catalog_manager.cc:1347] Generated new cluster ID: 0f7bacb7758348458bf51f07d547e6e0
I20260619 08:16:53.284787 16325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:16:53.288285 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.277437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44462 (local address 127.2.119.254:45921)
0619 08:16:53.277656 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.277664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.277688 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:16:53.278294 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.278301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.278347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.278459 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:53.279289 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.280292 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.281436 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.282101 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.282217 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.285096 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:53.285100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:53.285100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:53.285111 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:53.286791 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.287159 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.287162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.287162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.287268 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.287481 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.287482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.287482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.287710 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:53.287887 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.287953 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.288100 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":128,"threads_started":1}
I20260619 08:16:53.298782 16325 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:16:53.299449 16325 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:53.306211 16325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Generated new TSK 0
I20260619 08:16:53.306409 16325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:16:53.338162 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:53.429250 16340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:53.429476 16340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:53.429507 16340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:53.430660 16340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:53.430699 16340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:53.430725 16340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:16:53.432155 16340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:53.432408 16340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:53.432649 16340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:53.437911 16345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.437947 16346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.438069 16340 server_base.cc:1034] running on GCE node
W20260619 08:16:53.438242 16348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.438515 16340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:53.438717 16340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:53.439857 16340 hybrid_clock.cc:648] HybridClock initialized: now 1781857013439850 us; error 36 us; skew 500 ppm
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:53.442802 16340 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:16:53.443269 16340 webserver.cc:460] Webserver started at http://127.2.119.193:42809/ using document root <none> and password file <none>
I20260619 08:16:53.443578 16340 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:53.443662 16340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:53.443813 16340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:53.445297 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "09f6a9a368e5409086dc7407f9f2fa17"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "30522088741e94c2987605caf123b0f1"
server_key_iv: "fe2f543fd8967092c11603b4e52dbfb2"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.445783 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "09f6a9a368e5409086dc7407f9f2fa17"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "30522088741e94c2987605caf123b0f1"
server_key_iv: "fe2f543fd8967092c11603b4e52dbfb2"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.447010 16340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:53.447692 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.447844 16340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:16:53.447925 16340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "09f6a9a368e5409086dc7407f9f2fa17"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "30522088741e94c2987605caf123b0f1"
server_key_iv: "fe2f543fd8967092c11603b4e52dbfb2"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.447983 16340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:53.460865 16340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:53.463107 16340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:53.463220 16340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:53.463447 16340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:53.463776 16340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:53.463810 16340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.463832 16340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:53.463845 16340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.469635 16340 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:44627
I20260619 08:16:53.469736 16468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:44627 every 8 connection(s)
I20260619 08:16:53.470080 16340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:16:53.473608 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16340
I20260619 08:16:53.473695 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:16:53.473896 2527 external_mini_cluster.cc:1428] Setting key 1a780aa25e34bee8b25c2fe0db099adb
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:53.481012 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.470843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:53919 (local address 127.2.119.254:45921)
0619 08:16:53.470920 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.470922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.471119 (+ 197us) server_negotiation.cc:407] Connection header received
0619 08:16:53.472134 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.472136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.472146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.472200 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:53.472771 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.473285 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.474620 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.474824 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.474908 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.477504 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:53.477505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:53.477505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:53.477512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:53.479182 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.479736 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.479737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.479738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.479798 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.480183 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.480184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.480184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.480307 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:53.480378 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.480819 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.480894 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:16:53.481282 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.470791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:53919)
0619 08:16:53.470985 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.471034 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.472015 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.472256 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.472259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.472506 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:53.472695 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.472700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.473548 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.473549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.474518 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.474521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.474924 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.474925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.475056 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.475588 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:53.475596 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:53.477375 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:53.479273 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.479281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.479284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.479635 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.479936 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.479937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.479938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.480081 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.480386 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:53.480392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:53.480574 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.480702 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.480969 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260619 08:16:53.481942 16469 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:53.482059 16469 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:53.482270 16469 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:16:53.483043 16278 ts_manager.cc:194] Registered new tserver with Master: 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627)
I20260619 08:16:53.483690 16278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:53919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:16:53.504052 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:53.586987 16474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:53.587191 16474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:53.587215 16474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:53.588393 16474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:53.588438 16474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:53.588466 16474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:16:53.590047 16474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:53.590305 16474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:53.590564 16474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:53.595839 16479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.595976 16482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.595842 16480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.596100 16474 server_base.cc:1034] running on GCE node
I20260619 08:16:53.596302 16474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:53.596566 16474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:53.597783 16474 hybrid_clock.cc:648] HybridClock initialized: now 1781857013597765 us; error 34 us; skew 500 ppm
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:53.600821 16474 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:16:53.601380 16474 webserver.cc:460] Webserver started at http://127.2.119.194:35485/ using document root <none> and password file <none>
I20260619 08:16:53.601613 16474 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:53.601660 16474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:53.601756 16474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:53.602847 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "cf678f43c26a41cb8fac51d479cab6a7"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "fae2273df2bf81f7e74fdec26539b66b"
server_key_iv: "945946d9823329afc818a98557896d02"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.603170 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "cf678f43c26a41cb8fac51d479cab6a7"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "fae2273df2bf81f7e74fdec26539b66b"
server_key_iv: "945946d9823329afc818a98557896d02"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.604642 16474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:16:53.605551 16489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.605736 16474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:53.605901 16474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "cf678f43c26a41cb8fac51d479cab6a7"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "fae2273df2bf81f7e74fdec26539b66b"
server_key_iv: "945946d9823329afc818a98557896d02"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.605966 16474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:53.618639 16474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:53.620939 16474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:53.621104 16474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:53.621367 16474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:53.621673 16474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:53.621708 16474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.621731 16474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:53.621743 16474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.627576 16474 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:41215
I20260619 08:16:53.627712 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:41215 every 8 connection(s)
I20260619 08:16:53.627974 16474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:16:53.629410 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16474
I20260619 08:16:53.629511 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:16:53.629707 2527 external_mini_cluster.cc:1428] Setting key d0c80d17d895abddcd65f4e84f139c41
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:16:53.638736 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.628732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:58519 (local address 127.2.119.254:45921)
0619 08:16:53.628801 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.628804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.629027 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:16:53.629798 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.629799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.629812 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.629895 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:53.631172 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.631736 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.632751 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.632974 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.633035 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.635694 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:53.635696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:53.635697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:53.635704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:53.637167 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.637678 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.637680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.637680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.637747 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.638066 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.638067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.638067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.638189 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:53.638262 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.638557 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.638645 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:16:53.638914 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.628697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:58519)
0619 08:16:53.628901 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.628936 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.629657 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.629956 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.629960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.630789 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:53.631074 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.631080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.631832 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.631833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.632659 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.632661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.633098 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.633099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.633223 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.633808 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:53.633817 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:53.635591 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:53.637279 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.637286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.637288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.637591 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.637817 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.637817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.637818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.637981 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.638264 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:53.638271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:53.638418 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.638524 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.638680 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":101,"threads_started":1}
I20260619 08:16:53.639472 16603 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:53.639595 16603 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:53.639778 16603 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
I20260619 08:16:53.640169 16278 ts_manager.cc:194] Registered new tserver with Master: cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:53.640599 16278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:58519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:16:53.656850 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:53.738332 16608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:53.738549 16608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:53.738574 16608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:53.739677 16608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:53.739714 16608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:53.739739 16608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:16:53.741199 16608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:53.741456 16608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:53.741645 16608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:53.747169 16616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.747176 16614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.747295 16608 server_base.cc:1034] running on GCE node
W20260619 08:16:53.747164 16613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.747558 16608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:53.747824 16608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:53.749001 16608 hybrid_clock.cc:648] HybridClock initialized: now 1781857013748997 us; error 38 us; skew 500 ppm
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:53.751750 16608 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:16:53.752235 16608 webserver.cc:460] Webserver started at http://127.2.119.195:44167/ using document root <none> and password file <none>
I20260619 08:16:53.752472 16608 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:53.752547 16608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:53.752684 16608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:16:53.753816 16608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "d63357d668da41fe9e08d192624e0932"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "49d5b16a4c63c331dd77208e1ac60cc2"
server_key_iv: "27fb6149d380143a8eb75fb0c13606a0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.754180 16608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "d63357d668da41fe9e08d192624e0932"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "49d5b16a4c63c331dd77208e1ac60cc2"
server_key_iv: "27fb6149d380143a8eb75fb0c13606a0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.755620 16608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:16:53.756419 16623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.756646 16608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:53.756752 16608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "d63357d668da41fe9e08d192624e0932"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "49d5b16a4c63c331dd77208e1ac60cc2"
server_key_iv: "27fb6149d380143a8eb75fb0c13606a0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.756829 16608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:53.764205 16608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:53.766264 16608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:53.766428 16608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:53.766678 16608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:53.767004 16608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:53.767061 16608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.767093 16608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:53.767122 16608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.773447 16608 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:36059
I20260619 08:16:53.773492 16736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:36059 every 8 connection(s)
I20260619 08:16:53.773823 16608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:53.781915 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16608
I20260619 08:16:53.782066 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:16:53.782272 2527 external_mini_cluster.cc:1428] Setting key 63ff9b406649e91bf75d0aa430ec26e8
I20260619 08:16:53.784220 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.774483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:48849 (local address 127.2.119.254:45921)
0619 08:16:53.774527 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.774529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.774731 (+ 202us) server_negotiation.cc:407] Connection header received
0619 08:16:53.775261 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.775262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.775275 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.775341 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:53.776310 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.776830 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.777928 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.778137 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.778222 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.780503 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:53.780505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:53.780506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:53.780521 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:53.782563 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.783110 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.783111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.783112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.783184 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.783475 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.783476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.783476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.783611 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:53.783696 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.783997 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.784111 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:16:53.784466 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.774464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:48849)
0619 08:16:53.774603 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.774644 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.775154 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.775411 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.775416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.776005 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:53.776208 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.776212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.776927 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.776928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.777823 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.777825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.778232 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.778233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.778358 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.778861 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:53.778870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:53.780376 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:53.782688 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.782697 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.782699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.783006 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.783259 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:53.783259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:53.783260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:53.783379 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:53.783686 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:53.783693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:53.783848 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.783972 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.784192 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260619 08:16:53.785012 16737 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:53.785101 16737 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:53.785250 16737 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
I20260619 08:16:53.785670 16278 ts_manager.cc:194] Registered new tserver with Master: d63357d668da41fe9e08d192624e0932 (127.2.119.195:36059)
I20260619 08:16:53.786060 16278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:48849
I20260619 08:16:53.794463 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:16:53.802860 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.796007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.2.119.254:45921)
0619 08:16:53.796075 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.796078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.796194 (+ 116us) server_negotiation.cc:407] Connection header received
0619 08:16:53.796223 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.796223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.796233 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.796294 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:53.796845 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.797488 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.798290 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.798487 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.798706 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.800060 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:53.800062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:53.800062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:53.800071 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:53.801639 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.801996 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.801997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.801998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.802063 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:53.802275 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:53.802276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:53.802276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:53.802440 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:53.802558 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.802671 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.802750 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:16:53.804596 16278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:16:53.806346 16278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:16:53.814299 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.0.0.1:60296)
0619 08:16:53.809418 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.809461 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.809537 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.809877 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.809880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.809882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:53.809952 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.809954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.811396 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.811397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.813005 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.813009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.813937 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.813938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.814038 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.814054 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.814136 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.814205 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":95,"thread_start_us":47,"threads_started":1}
I20260619 08:16:53.814464 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:36059 (local address 127.0.0.1:59716)
0619 08:16:53.809650 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.809662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.809703 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.810176 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.810177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.810179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:53.810242 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.810245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.811317 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.811318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.813005 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.813009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.814227 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.814229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.814301 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.814305 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.814338 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.814385 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":80,"thread_start_us":124,"threads_started":1}
I20260619 08:16:53.814675 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.2.119.193:44627)
0619 08:16:53.809646 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.809651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.809662 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:16:53.809704 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.809754 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.809774 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.809856 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:53.810091 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.811267 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.813106 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.813836 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.813915 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.813973 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.814528 (+ 555us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.814577 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260619 08:16:53.815205 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.2.119.195:36059)
0619 08:16:53.809842 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.809846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.809857 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:16:53.810045 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.810047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.810066 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.810176 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:53.810330 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.811201 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.813386 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.814128 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.814976 (+ 848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.815057 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.815084 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.815127 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260619 08:16:53.815519 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41215 (local address 127.0.0.1:38942)
0619 08:16:53.809727 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.809881 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.809967 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.810279 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.810280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.810282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:53.810320 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.810322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.813139 (+ 2817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.813139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.814144 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.814147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.815199 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.815201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.815278 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.815283 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.815391 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.815449 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":143,"threads_started":1}
I20260619 08:16:53.815433 16403 tablet_service.cc:1467] Processing CreateTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 (DEFAULT_TABLE table=test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:53.815791 16403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c357c13b7c2e45a79d2bbd97fc3c7832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:53.815783 16671 tablet_service.cc:1467] Processing CreateTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 (DEFAULT_TABLE table=test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:53.815925 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.809428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.2.119.194:41215)
0619 08:16:53.809683 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.809688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.809937 (+ 249us) server_negotiation.cc:407] Connection header received
0619 08:16:53.810144 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.810148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.810167 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.810241 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:53.810460 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.811540 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.814241 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.815096 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.815196 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.815255 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.815746 (+ 491us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.815798 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":124,"threads_started":1}
I20260619 08:16:53.816133 16671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c357c13b7c2e45a79d2bbd97fc3c7832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:53.816563 16537 tablet_service.cc:1467] Processing CreateTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 (DEFAULT_TABLE table=test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:53.816891 16537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c357c13b7c2e45a79d2bbd97fc3c7832. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:53.819731 16754 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Bootstrap starting.
I20260619 08:16:53.820099 16755 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Bootstrap starting.
I20260619 08:16:53.820603 16754 tablet_bootstrap.cc:654] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:53.820781 16755 tablet_bootstrap.cc:654] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:53.821205 16754 log.cc:824] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:53.821285 16755 log.cc:824] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:53.821671 16756 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Bootstrap starting.
I20260619 08:16:53.822113 16755 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: No bootstrap required, opened a new log
I20260619 08:16:53.822208 16755 ts_tablet_manager.cc:1397] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:16:53.822371 16756 tablet_bootstrap.cc:654] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:53.822923 16756 log.cc:824] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:53.823477 16754 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: No bootstrap required, opened a new log
I20260619 08:16:53.823554 16754 ts_tablet_manager.cc:1397] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:16:53.823840 16756 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: No bootstrap required, opened a new log
I20260619 08:16:53.823911 16756 ts_tablet_manager.cc:1397] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:16:53.824193 16755 raft_consensus.cc:348] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.824363 16755 raft_consensus.cc:374] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:53.824405 16755 raft_consensus.cc:729] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f6a9a368e5409086dc7407f9f2fa17, State: Initialized, Role: FOLLOWER
I20260619 08:16:53.824529 16755 consensus_queue.cc:260] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.824826 16755 ts_tablet_manager.cc:1428] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:16:53.824999 16469 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
I20260619 08:16:53.825855 16754 raft_consensus.cc:348] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.826049 16754 raft_consensus.cc:374] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:53.826092 16754 raft_consensus.cc:729] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf678f43c26a41cb8fac51d479cab6a7, State: Initialized, Role: FOLLOWER
I20260619 08:16:53.826092 16756 raft_consensus.cc:348] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.826215 16754 consensus_queue.cc:260] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.826261 16756 raft_consensus.cc:374] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:53.826303 16756 raft_consensus.cc:729] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63357d668da41fe9e08d192624e0932, State: Initialized, Role: FOLLOWER
I20260619 08:16:53.826539 16754 ts_tablet_manager.cc:1428] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:16:53.826491 16756 consensus_queue.cc:260] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.826574 16603 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
I20260619 08:16:53.826798 16756 ts_tablet_manager.cc:1428] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:16:53.826838 16737 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
I20260619 08:16:53.828722 16762 raft_consensus.cc:482] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:16:53.828831 16762 raft_consensus.cc:504] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.829231 16762 leader_election.cc:288] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627), cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:53.833118 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.829658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:37459 (local address 127.2.119.193:44627)
0619 08:16:53.829698 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.829700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.829707 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:16:53.829729 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.829730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.829739 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.829786 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:53.829919 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.830599 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832134 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.832717 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832804 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.832813 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.833000 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.833049 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:16:53.833153 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.829408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41215 (local address 127.2.119.195:59947)
0619 08:16:53.829488 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.829501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.829541 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.829705 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.829706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.829708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:53.829774 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.829777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.830687 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.830688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.831999 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.832002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.832824 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.832976 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.832980 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.833028 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.833056 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":86}
I20260619 08:16:53.833176 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.829428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.2.119.195:37459)
0619 08:16:53.829567 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.829576 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:53.829618 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:53.829789 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:53.829790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:53.829791 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:53.829861 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.829862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.830687 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.830688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.831999 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:53.832002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.832824 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:53.832907 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.832912 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:16:53.832980 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 08:16:53.833034 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":93,"thread_start_us":94,"threads_started":1}
I20260619 08:16:53.833536 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.829463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:59947 (local address 127.2.119.194:41215)
0619 08:16:53.829511 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.829513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.829527 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:16:53.829609 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.829610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.829619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.829700 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:53.829852 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.830606 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832134 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.832730 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.832804 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.832813 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.833423 (+ 610us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.833473 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:53.833508 16423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c357c13b7c2e45a79d2bbd97fc3c7832" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09f6a9a368e5409086dc7407f9f2fa17" is_pre_election: true
I20260619 08:16:53.833665 16423 raft_consensus.cc:2454] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 0.
I20260619 08:16:53.834038 16626 leader_election.cc:302] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09f6a9a368e5409086dc7407f9f2fa17, d63357d668da41fe9e08d192624e0932; no voters:
I20260619 08:16:53.834076 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c357c13b7c2e45a79d2bbd97fc3c7832" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf678f43c26a41cb8fac51d479cab6a7" is_pre_election: true
I20260619 08:16:53.834204 16557 raft_consensus.cc:2454] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 0.
I20260619 08:16:53.834185 16762 raft_consensus.cc:2781] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:16:53.834247 16762 raft_consensus.cc:482] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:16:53.834285 16762 raft_consensus.cc:3037] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:53.834972 16762 raft_consensus.cc:504] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.835116 16762 leader_election.cc:288] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 election: Requested vote from peers 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627), cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:53.835294 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c357c13b7c2e45a79d2bbd97fc3c7832" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf678f43c26a41cb8fac51d479cab6a7"
I20260619 08:16:53.835332 16423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c357c13b7c2e45a79d2bbd97fc3c7832" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09f6a9a368e5409086dc7407f9f2fa17"
I20260619 08:16:53.835412 16423 raft_consensus.cc:3037] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:53.835391 16557 raft_consensus.cc:3037] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:53.836086 16557 raft_consensus.cc:2454] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 1.
I20260619 08:16:53.836099 16423 raft_consensus.cc:2454] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 1.
I20260619 08:16:53.836263 16627 leader_election.cc:302] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf678f43c26a41cb8fac51d479cab6a7, d63357d668da41fe9e08d192624e0932; no voters:
I20260619 08:16:53.836355 16762 raft_consensus.cc:2781] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:53.836540 16762 raft_consensus.cc:686] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 1 LEADER]: Becoming Leader. State: Replica: d63357d668da41fe9e08d192624e0932, State: Running, Role: LEADER
I20260619 08:16:53.836648 16762 consensus_queue.cc:237] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:53.837499 16278 catalog_manager.cc:5495] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63357d668da41fe9e08d192624e0932 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "d63357d668da41fe9e08d192624e0932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } health_report { overall_health: UNKNOWN } } }
I20260619 08:16:53.843570 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.841200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.2.119.195:36059)
0619 08:16:53.841250 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:53.841252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:53.841261 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:16:53.841319 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:53.841320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:53.841339 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:53.841420 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:16:53.841573 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.842044 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.842797 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:53.843007 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:53.843081 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:53.843234 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:16:53.843335 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:16:53.843421 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:53.843460 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 08:16:53.843491 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:16:53.845288 16557 raft_consensus.cc:1261] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Refusing update from remote peer d63357d668da41fe9e08d192624e0932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:53.845288 16423 raft_consensus.cc:1261] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Refusing update from remote peer d63357d668da41fe9e08d192624e0932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:53.845702 16762 consensus_queue.cc:1035] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:53.845875 16764 consensus_queue.cc:1035] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:53.848143 16767 mvcc.cc:205] Tried to move back new op lower bound from 7298486328707981312 to 7298486328675631104. Current Snapshot: MvccSnapshot[applied={T|T < 7298486328707981312}]
I20260619 08:16:53.848565 16768 mvcc.cc:205] Tried to move back new op lower bound from 7298486328707981312 to 7298486328675631104. Current Snapshot: MvccSnapshot[applied={T|T < 7298486328707981312}]
I20260619 08:16:53.849406 16769 mvcc.cc:205] Tried to move back new op lower bound from 7298486328707981312 to 7298486328675631104. Current Snapshot: MvccSnapshot[applied={T|T < 7298486328707981312 or (T in {7298486328707981312})}]
I20260619 08:16:53.850639 16277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:53.850721 16277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:53.851598 16277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Sending DeleteTablet for 3 replicas of tablet c357c13b7c2e45a79d2bbd97fc3c7832
I20260619 08:16:53.851945 16671 tablet_service.cc:1514] Processing DeleteTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:53 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:59716
I20260619 08:16:53.851955 16537 tablet_service.cc:1514] Processing DeleteTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:53 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:38942
I20260619 08:16:53.852000 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16340
I20260619 08:16:53.852171 16403 tablet_service.cc:1514] Processing DeleteTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:53 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:60296
I20260619 08:16:53.852691 16776 tablet_replica.cc:330] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: stopping tablet replica
I20260619 08:16:53.852800 16776 raft_consensus.cc:2229] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:16:53.852905 16776 raft_consensus.cc:2258] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:53.853435 16776 ts_tablet_manager.cc:1905] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:53.855116 16776 ts_tablet_manager.cc:1918] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:53.855180 16776 log.cc:1196] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/c357c13b7c2e45a79d2bbd97fc3c7832
I20260619 08:16:53.855454 16776 ts_tablet_manager.cc:1939] T c357c13b7c2e45a79d2bbd97fc3c7832 P d63357d668da41fe9e08d192624e0932: Deleting consensus metadata
I20260619 08:16:53.855924 16262 catalog_manager.cc:4841] TS d63357d668da41fe9e08d192624e0932 (127.2.119.195:36059): tablet c357c13b7c2e45a79d2bbd97fc3c7832 (table test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]) successfully deleted
W20260619 08:16:53.857335 16264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:44627 (error 108)
I20260619 08:16:53.857578 16777 tablet_replica.cc:330] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: stopping tablet replica
I20260619 08:16:53.857652 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16474
I20260619 08:16:53.857759 16777 raft_consensus.cc:2229] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:53.857889 16777 raft_consensus.cc:2258] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:53.863750 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16608
I20260619 08:16:53.863883 16750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.863727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.0.0.1:60298)
0619 08:16:53.863800 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.863842 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 08:16:53.863955 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:53.863729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41215 (local address 127.0.0.1:38952)
0619 08:16:53.863856 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:53.863882 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:41215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":36}
W20260619 08:16:53.864708 16265 catalog_manager.cc:4584] TS cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c357c13b7c2e45a79d2bbd97fc3c7832: Network error: Client connection negotiation failed: client connection to 127.2.119.194:41215: connect: Connection refused (error 111)
I20260619 08:16:53.868507 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16246
I20260619 08:16:53.873939 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--webserver_interface=127.2.119.254
--webserver_port=40869
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:53.955476 16779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:53.955673 16779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:53.955695 16779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:53.956892 16779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:53.956935 16779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:53.956950 16779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:53.956964 16779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:53.956983 16779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:53.958511 16779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:53.958757 16779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:53.959020 16779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:16:53.968071 16779 server_base.cc:1034] running on GCE node
W20260619 08:16:53.968037 16788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.968078 16785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:53.968037 16786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:53.968581 16779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:53.968910 16779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:53.970077 16779 hybrid_clock.cc:648] HybridClock initialized: now 1781857013970045 us; error 52 us; skew 500 ppm
Jun 19 08:16:53 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:53.972779 16779 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:53.973335 16779 webserver.cc:460] Webserver started at http://127.2.119.254:40869/ using document root <none> and password file <none>
I20260619 08:16:53.973601 16779 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:53.973688 16779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:53.975126 16779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:16:53.976030 16795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:53.976235 16779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:53.976362 16779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ca02f7da742743e5b7d20b4c704e2143"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "abe8b3c8918c8a9d8bcf28ff4c71f2ff"
server_key_iv: "f03b4987ddc438ec98db36328240fea0"
server_key_version: "encryptionkey@0"
I20260619 08:16:53.976773 16779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:53.985857 16779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:53.988076 16779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:53.988184 16779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:53.992350 16779 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45921
I20260619 08:16:53.992518 16847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45921 every 8 connection(s)
I20260619 08:16:53.992800 16779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:53.994488 16848 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:16:53.995211 16848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap starting.
I20260619 08:16:53.998735 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16779
I20260619 08:16:53.999018 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:44627
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=42809
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:54.004046 16848 log.cc:824] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:54.006100 16848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:16:54.006402 16848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap complete.
I20260619 08:16:54.008730 16848 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.008960 16848 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.009130 16848 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.009187 16848 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:54.009255 16848 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:54.009326 16848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:16:54.010337 16848 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.010458 16848 leader_election.cc:302] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca02f7da742743e5b7d20b4c704e2143; no voters:
I20260619 08:16:54.010671 16848 leader_election.cc:288] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:16:54.010748 16852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:16:54.010883 16852 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 LEADER]: Becoming Leader. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Running, Role: LEADER
I20260619 08:16:54.010993 16848 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:54.010979 16852 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.011384 16854 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca02f7da742743e5b7d20b4c704e2143. Latest consensus state: current_term: 2 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:54.011529 16854 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:54.011714 16853 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:54.011817 16853 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:54.011904 16861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:54.012735 16861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:54.012837 16861 catalog_manager.cc:1259] Loaded cluster ID: 0f7bacb7758348458bf51f07d547e6e0
I20260619 08:16:54.012874 16861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:16:54.013185 16861 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 08:16:54.015909 16861 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 08:16:54.018318 16861 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:16:54.018527 16861 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Loaded TSK: 0
I20260619 08:16:54.018738 16861 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:16:54.093654 16850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.093889 16850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.093943 16850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.095074 16850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.095140 16850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.095187 16850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:16:54.096732 16850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:44627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=42809
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.097002 16850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.097268 16850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.102614 16875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.102746 16878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.102623 16876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.102838 16850 server_base.cc:1034] running on GCE node
I20260619 08:16:54.103271 16850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.103546 16850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.104725 16850 hybrid_clock.cc:648] HybridClock initialized: now 1781857014104696 us; error 40 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.107704 16850 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:16:54.108278 16850 webserver.cc:460] Webserver started at http://127.2.119.193:42809/ using document root <none> and password file <none>
I20260619 08:16:54.108589 16850 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.108688 16850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.110644 16850 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.111604 16885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.111820 16850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:16:54.111927 16850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "09f6a9a368e5409086dc7407f9f2fa17"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "30522088741e94c2987605caf123b0f1"
server_key_iv: "fe2f543fd8967092c11603b4e52dbfb2"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.112452 16850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.135171 16850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.137950 16850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.138123 16850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.138417 16850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:54.138947 16892 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:16:54.140517 16850 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:16:54.140591 16850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:16:54.140628 16850 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:16:54.141638 16850 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:16:54.141690 16850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.141762 16892 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Bootstrap starting.
I20260619 08:16:54.149395 16850 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:44627
I20260619 08:16:54.149636 16999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:44627 every 8 connection(s)
I20260619 08:16:54.149871 16850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:16:54.152422 16892 log.cc:824] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:54.154865 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 16850
I20260619 08:16:54.155097 16892 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:16:54.155171 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:41215
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=35485
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:54.155510 16892 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Bootstrap complete.
I20260619 08:16:54.156756 16892 ts_tablet_manager.cc:1397] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.006s
I20260619 08:16:54.158223 16892 raft_consensus.cc:348] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.158631 16892 raft_consensus.cc:729] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f6a9a368e5409086dc7407f9f2fa17, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.158807 16892 consensus_queue.cc:260] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.159080 16892 ts_tablet_manager.cc:1428] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:54.164836 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.150688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:43609 (local address 127.2.119.254:45921)
0619 08:16:54.152721 (+ 2033us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.152729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.152757 (+ 28us) server_negotiation.cc:407] Connection header received
0619 08:16:54.152859 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.152864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.152904 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.153059 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:54.153971 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.155374 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.157055 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.157657 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.159271 (+ 1614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.160371 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:54.160375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:54.160376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:54.160397 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:54.162635 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.163203 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.163206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.163207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.163335 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.163671 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.163672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.163673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.163977 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:54.164071 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.164458 (+ 387us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.164604 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1989,"thread_start_us":69,"threads_started":1}
I20260619 08:16:54.164834 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.150582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:43609)
0619 08:16:54.150798 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.150906 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.151564 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.153095 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.153100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.153546 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:54.153828 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.153835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.155506 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.155507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.156945 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.156948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.157782 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.157783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.157914 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.158530 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:54.158540 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:54.160257 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:54.162738 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.162746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.162748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.163097 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.163436 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.163438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.163438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.163588 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.164067 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:54.164075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:54.164249 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.164378 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.164569 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260619 08:16:54.166288 17000 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:54.166404 17000 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:54.166606 17000 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
I20260619 08:16:54.167455 16812 ts_manager.cc:194] Registered new tserver with Master: 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627)
I20260619 08:16:54.168735 16812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:43609
I20260619 08:16:54.169787 17000 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
I20260619 08:16:54.174721 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.169023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.2.119.193:44627)
0619 08:16:54.169194 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.169199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.169604 (+ 405us) server_negotiation.cc:407] Connection header received
0619 08:16:54.169643 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.169645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.169660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.169725 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:54.170084 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.171353 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.173489 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.174276 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.174360 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.174427 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.174598 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.174641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260619 08:16:54.174742 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.168132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.0.0.1:60300)
0619 08:16:54.169405 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.169444 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.169521 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.169788 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.169791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.169793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:54.170011 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.170014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.171459 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.171460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.173400 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.173403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.174361 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.174362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.174461 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.174478 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.174574 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.174637 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"thread_start_us":91,"threads_started":1}
I20260619 08:16:54.175510 16934 tablet_service.cc:1514] Processing DeleteTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:53 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:60300
I20260619 08:16:54.176273 17009 tablet_replica.cc:330] stopping tablet replica
I20260619 08:16:54.176421 17009 raft_consensus.cc:2229] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:54.176527 17009 raft_consensus.cc:2258] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:54.176909 17009 ts_tablet_manager.cc:1905] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.179345 17009 ts_tablet_manager.cc:1918] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.179453 17009 log.cc:1196] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/c357c13b7c2e45a79d2bbd97fc3c7832
I20260619 08:16:54.179721 17009 ts_tablet_manager.cc:1939] T c357c13b7c2e45a79d2bbd97fc3c7832 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting consensus metadata
I20260619 08:16:54.180248 16798 catalog_manager.cc:4841] TS 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627): tablet c357c13b7c2e45a79d2bbd97fc3c7832 (table test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]) successfully deleted
W20260619 08:16:54.244920 17005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.245168 17005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.245205 17005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.246389 17005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.246461 17005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.246506 17005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:16:54.248004 17005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:41215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=35485
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.248286 17005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.248520 17005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.253669 17014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.253661 17017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.253875 17015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.254154 17005 server_base.cc:1034] running on GCE node
I20260619 08:16:54.254393 17005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.254670 17005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.255859 17005 hybrid_clock.cc:648] HybridClock initialized: now 1781857014255842 us; error 29 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.258599 17005 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:16:54.259055 17005 webserver.cc:460] Webserver started at http://127.2.119.194:35485/ using document root <none> and password file <none>
I20260619 08:16:54.259258 17005 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.259313 17005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.260710 17005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:54.261467 17024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.261650 17005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:54.261739 17005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "cf678f43c26a41cb8fac51d479cab6a7"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "fae2273df2bf81f7e74fdec26539b66b"
server_key_iv: "945946d9823329afc818a98557896d02"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.262056 17005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.274541 17005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.276793 17005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.276903 17005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.277074 17005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:54.277669 17031 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:16:54.279155 17005 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:16:54.279207 17005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:16:54.279233 17005 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:16:54.279911 17005 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:16:54.279969 17005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:16:54.280136 17031 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Bootstrap starting.
I20260619 08:16:54.287002 17005 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:41215
I20260619 08:16:54.287109 17138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:41215 every 8 connection(s)
I20260619 08:16:54.287442 17005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:16:54.291993 17031 log.cc:824] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:54.292712 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17005
I20260619 08:16:54.293033 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:36059
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=44167
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:54.295130 17031 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:16:54.295517 17031 tablet_bootstrap.cc:492] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Bootstrap complete.
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:54.296793 17031 ts_tablet_manager.cc:1397] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260619 08:16:54.298041 17031 raft_consensus.cc:348] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.298326 17031 raft_consensus.cc:729] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf678f43c26a41cb8fac51d479cab6a7, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.298501 17031 consensus_queue.cc:260] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.298866 17031 ts_tablet_manager.cc:1428] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:16:54.299281 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.288328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:34265 (local address 127.2.119.254:45921)
0619 08:16:54.288394 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.288397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.288677 (+ 280us) server_negotiation.cc:407] Connection header received
0619 08:16:54.289541 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.289542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.289553 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.289618 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:54.290719 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.291264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.292904 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.293250 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.293346 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.296172 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:54.296173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:54.296174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:54.296182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:54.297720 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.298151 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.298153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.298153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.298216 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.298512 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.298513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.298513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.298676 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:54.298738 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.299087 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.299189 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 08:16:54.299476 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.288225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:34265)
0619 08:16:54.288495 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.288553 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.289407 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.289695 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.289701 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.290197 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:54.290582 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.290590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.291395 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.291396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.292750 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.292753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.293438 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.293440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.293625 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.294296 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:54.294307 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:54.295951 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:54.297820 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.297827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.297829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.298069 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.298285 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.298286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.298287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.298446 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.298744 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:54.298751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:54.298938 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.299054 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.299217 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
I20260619 08:16:54.299913 17139 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:54.300027 17139 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:54.300225 17139 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
I20260619 08:16:54.300942 16812 ts_manager.cc:194] Registered new tserver with Master: cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:54.301496 16812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:34265
I20260619 08:16:54.302443 17139 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857013, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.194@KRBTEST.COM
I20260619 08:16:54.309300 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.301337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41215 (local address 127.0.0.1:38956)
0619 08:16:54.301650 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.301661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.301702 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.301888 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.301889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.302186 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:54.302223 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.302225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.303306 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.303307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.303808 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.303810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.304307 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.304308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.304375 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.304960 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:54.304964 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:54.306319 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:54.308112 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.308120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.308122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.308322 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.308577 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.308578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.308578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.308646 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.308930 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:54.308937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:54.308989 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.309089 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.309199 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296}
I20260619 08:16:54.309324 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.301416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.2.119.194:41215)
0619 08:16:54.301583 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.301588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.301755 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:16:54.301796 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.301797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.301811 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.301887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:54.302309 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.303224 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.303907 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.304211 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.304303 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.306411 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:54.306419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:54.306420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:54.306427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:54.308007 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.308416 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.308419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.308419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.308502 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.308711 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.308712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.308712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.308856 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:54.308952 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.309125 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.309213 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260619 08:16:54.309937 17073 tablet_service.cc:1514] Processing DeleteTablet for tablet c357c13b7c2e45a79d2bbd97fc3c7832 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:53 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:38956
I20260619 08:16:54.310462 17146 tablet_replica.cc:330] stopping tablet replica
I20260619 08:16:54.310603 17146 raft_consensus.cc:2229] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:54.310680 17146 raft_consensus.cc:2258] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:54.311012 17146 ts_tablet_manager.cc:1905] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.313588 17146 ts_tablet_manager.cc:1918] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.313673 17146 log.cc:1196] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/c357c13b7c2e45a79d2bbd97fc3c7832
I20260619 08:16:54.313921 17146 ts_tablet_manager.cc:1939] T c357c13b7c2e45a79d2bbd97fc3c7832 P cf678f43c26a41cb8fac51d479cab6a7: Deleting consensus metadata
I20260619 08:16:54.314468 16799 catalog_manager.cc:4841] TS cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215): tablet c357c13b7c2e45a79d2bbd97fc3c7832 (table test-table [id=1e4aa2b4db1d499eb2c5ce92059cb5ca]) successfully deleted
W20260619 08:16:54.384095 17143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.384305 17143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.384328 17143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.385440 17143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.385474 17143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.385500 17143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:16:54.386847 17143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:36059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=44167
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.387027 17143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.387204 17143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.392228 17151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.392253 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.392256 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.392592 17143 server_base.cc:1034] running on GCE node
I20260619 08:16:54.392788 17143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.392994 17143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.394184 17143 hybrid_clock.cc:648] HybridClock initialized: now 1781857014394172 us; error 40 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.396766 17143 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:16:54.397198 17143 webserver.cc:460] Webserver started at http://127.2.119.195:44167/ using document root <none> and password file <none>
I20260619 08:16:54.397464 17143 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.397548 17143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.398911 17143 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.399572 17161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.399760 17143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.399857 17143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "d63357d668da41fe9e08d192624e0932"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "49d5b16a4c63c331dd77208e1ac60cc2"
server_key_iv: "27fb6149d380143a8eb75fb0c13606a0"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.400225 17143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.417886 17143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.419927 17143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.420094 17143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.420336 17143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:54.420663 17143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:54.420720 17143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.420754 17143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:54.420781 17143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.426589 17143 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:36059
I20260619 08:16:54.426638 17274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:36059 every 8 connection(s)
I20260619 08:16:54.426985 17143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:16:54.432433 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17143
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:16:54.437673 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.427863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:57987 (local address 127.2.119.254:45921)
0619 08:16:54.427952 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.427954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.428137 (+ 183us) server_negotiation.cc:407] Connection header received
0619 08:16:54.428863 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.428864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.428875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.428947 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:54.430010 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.430544 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.431659 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.431856 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.431929 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.434445 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:54.434447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:54.434447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:54.434455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:54.436030 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.436455 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.436456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.436457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.436514 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.436775 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.436776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.436777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.436927 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:54.436996 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.437459 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.437560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:16:54.437785 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.427765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:57987)
0619 08:16:54.427994 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.428039 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.428742 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.428999 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.429002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.429733 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:16:54.429919 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.429924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.430666 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.430667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.431568 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.431571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.431977 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.431979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.432090 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.432729 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:16:54.432737 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:16:54.434326 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:16:54.436131 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.436138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.436140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.436375 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.436590 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:16:54.436591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:16:54.436591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:16:54.436706 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:16:54.437025 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:16:54.437033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:16:54.437215 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.437374 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.437526 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":91,"threads_started":1}
I20260619 08:16:54.438526 17275 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45921
I20260619 08:16:54.438616 17275 heartbeater.cc:464] Registering TS with master...
I20260619 08:16:54.438793 17275 heartbeater.cc:511] Master 127.2.119.254:45921 requested a full tablet report, sending...
I20260619 08:16:54.439164 16812 ts_manager.cc:194] Registered new tserver with Master: d63357d668da41fe9e08d192624e0932 (127.2.119.195:36059)
I20260619 08:16:54.439540 16812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:57987
I20260619 08:16:54.439697 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.432862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.2.119.254:45921)
0619 08:16:54.432971 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.432973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.433009 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:16:54.433079 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.433080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.433092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.433166 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:16:54.433781 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.434324 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.435040 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.435239 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.435325 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.436606 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:16:54.436607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:16:54.436608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:16:54.436616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:16:54.438268 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.438726 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.438728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.438728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.438789 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:16:54.439090 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:16:54.439091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:16:54.439091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:16:54.439253 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:16:54.439439 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.439481 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.439570 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260619 08:16:54.440376 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:16:54.443311 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.440748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.2.119.254:45921)
0619 08:16:54.440801 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.440803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.440883 (+ 80us) server_negotiation.cc:407] Connection header received
0619 08:16:54.440969 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.440969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.440995 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.441060 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:16:54.441144 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.441735 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.442481 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.442702 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.442763 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.442935 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:16:54.443028 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:16:54.443129 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.443228 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.443254 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 08:16:54.443583 16812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:16:54.445102 16812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:16:54.447350 16934 tablet_service.cc:1467] Processing CreateTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 (DEFAULT_TABLE table=test-table [id=a8dd7bff412c43d5a34d0676e18a49ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:54.447549 16934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9096c6c375984c4f95d60d7afa8a1e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:54.449196 16892 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Bootstrap starting.
I20260619 08:16:54.449415 17073 tablet_service.cc:1467] Processing CreateTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 (DEFAULT_TABLE table=test-table [id=a8dd7bff412c43d5a34d0676e18a49ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:54.449548 16892 tablet_bootstrap.cc:654] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:54.449581 17073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9096c6c375984c4f95d60d7afa8a1e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:54.450161 16892 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: No bootstrap required, opened a new log
I20260619 08:16:54.450217 16892 ts_tablet_manager.cc:1397] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.450318 16892 raft_consensus.cc:348] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.450433 16892 raft_consensus.cc:374] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:54.450487 16892 raft_consensus.cc:729] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f6a9a368e5409086dc7407f9f2fa17, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.450549 16892 consensus_queue.cc:260] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.450660 16892 ts_tablet_manager.cc:1428] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 08:16:54.451135 17031 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Bootstrap starting.
I20260619 08:16:54.451467 17031 tablet_bootstrap.cc:654] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:54.452241 17031 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: No bootstrap required, opened a new log
I20260619 08:16:54.452306 17031 ts_tablet_manager.cc:1397] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:16:54.452448 17031 raft_consensus.cc:348] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.452519 17031 raft_consensus.cc:374] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:54.452559 17031 raft_consensus.cc:729] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf678f43c26a41cb8fac51d479cab6a7, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.452616 17031 consensus_queue.cc:260] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.452710 17031 ts_tablet_manager.cc:1428] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.454489 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.447115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:36059 (local address 127.0.0.1:59742)
0619 08:16:54.451172 (+ 4057us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.451184 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.451230 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.451436 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.451437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.451439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:54.451511 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.451512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.452426 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.452427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.453576 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.453579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.454304 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.454305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.454361 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.454364 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.454407 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.454439 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4038}
I20260619 08:16:54.454511 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.447229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.2.119.195:36059)
0619 08:16:54.447855 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.447860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.451290 (+ 3430us) server_negotiation.cc:407] Connection header received
0619 08:16:54.451338 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.451341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.451359 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.451430 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:54.451601 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.452285 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.453647 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.454212 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.454288 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.454328 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.454419 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.454450 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":127,"threads_started":1}
I20260619 08:16:54.455160 17209 tablet_service.cc:1467] Processing CreateTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 (DEFAULT_TABLE table=test-table [id=a8dd7bff412c43d5a34d0676e18a49ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:16:54.455422 17209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9096c6c375984c4f95d60d7afa8a1e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:16:54.458248 17284 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Bootstrap starting.
I20260619 08:16:54.458843 17284 tablet_bootstrap.cc:654] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Neither blocks nor log segments found. Creating new log.
I20260619 08:16:54.459314 17284 log.cc:824] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:54.460152 17284 tablet_bootstrap.cc:492] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: No bootstrap required, opened a new log
I20260619 08:16:54.460234 17284 ts_tablet_manager.cc:1397] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:16:54.461642 17284 raft_consensus.cc:348] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.461800 17284 raft_consensus.cc:374] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:16:54.461843 17284 raft_consensus.cc:729] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63357d668da41fe9e08d192624e0932, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.461966 17284 consensus_queue.cc:260] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.462262 17284 ts_tablet_manager.cc:1428] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:16:54.462285 17275 heartbeater.cc:503] Master 127.2.119.254:45921 was elected leader, sending a full tablet report...
I20260619 08:16:54.530527 17286 raft_consensus.cc:482] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:16:54.530650 17286 raft_consensus.cc:504] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.531060 17286 leader_election.cc:288] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627), cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:54.534017 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.531256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.2.119.195:46187)
0619 08:16:54.531313 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.531327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.531371 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.531553 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.531554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.531556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:54.531648 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.531652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.532335 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.532337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.533120 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.533123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.533734 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.533822 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.533826 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.533880 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.533923 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:16:54.534035 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.531332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:46187 (local address 127.2.119.193:44627)
0619 08:16:54.531390 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.531392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.531400 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:16:54.531450 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.531451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.531464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.531535 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:54.531725 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.532255 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533214 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.533652 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533709 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.533717 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.533911 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.533955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:16:54.534153 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.531332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:51457 (local address 127.2.119.194:41215)
0619 08:16:54.531378 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.531381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.531541 (+ 160us) server_negotiation.cc:407] Connection header received
0619 08:16:54.531573 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.531575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.531602 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.531679 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:16:54.531851 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.532357 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533271 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.533756 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533827 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.533904 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.534019 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.534069 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:16:54.534300 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.531256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:41215 (local address 127.2.119.195:51457)
0619 08:16:54.531448 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.531459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.531526 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.531703 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:16:54.531704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:16:54.531706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:16:54.531779 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.531781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.532437 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.532437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.533181 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:16:54.533183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.533834 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.533835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:16:54.533940 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.533944 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:16:54.534003 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:16:54.534036 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":14,"thread_start_us":113,"threads_started":1}
I20260619 08:16:54.534368 16954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9096c6c375984c4f95d60d7afa8a1e96" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09f6a9a368e5409086dc7407f9f2fa17" is_pre_election: true
I20260619 08:16:54.534538 16954 raft_consensus.cc:2454] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 0.
I20260619 08:16:54.534601 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9096c6c375984c4f95d60d7afa8a1e96" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf678f43c26a41cb8fac51d479cab6a7" is_pre_election: true
I20260619 08:16:54.534752 17093 raft_consensus.cc:2454] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 0.
I20260619 08:16:54.534819 17164 leader_election.cc:302] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09f6a9a368e5409086dc7407f9f2fa17, d63357d668da41fe9e08d192624e0932; no voters:
I20260619 08:16:54.534986 17286 raft_consensus.cc:2781] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:16:54.535058 17286 raft_consensus.cc:482] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:16:54.535085 17286 raft_consensus.cc:3037] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:54.535812 17286 raft_consensus.cc:504] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
I20260619 08:16:54.535975 17286 leader_election.cc:288] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 election: Requested vote from peers 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627), cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215)
I20260619 08:16:54.536156 16954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9096c6c375984c4f95d60d7afa8a1e96" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09f6a9a368e5409086dc7407f9f2fa17"
I20260619 08:16:54.536144 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9096c6c375984c4f95d60d7afa8a1e96" candidate_uuid: "d63357d668da41fe9e08d192624e0932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf678f43c26a41cb8fac51d479cab6a7"
I20260619 08:16:54.536226 17093 raft_consensus.cc:3037] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:54.536244 16954 raft_consensus.cc:3037] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:16:54.536924 17093 raft_consensus.cc:2454] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 1.
I20260619 08:16:54.537115 17165 leader_election.cc:302] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf678f43c26a41cb8fac51d479cab6a7, d63357d668da41fe9e08d192624e0932; no voters:
I20260619 08:16:54.537231 17286 raft_consensus.cc:2781] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:16:54.537401 16954 raft_consensus.cc:2454] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63357d668da41fe9e08d192624e0932 in term 1.
I20260619 08:16:54.537456 17286 raft_consensus.cc:686] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 1 LEADER]: Becoming Leader. State: Replica: d63357d668da41fe9e08d192624e0932, State: Running, Role: LEADER
I20260619 08:16:54.537555 17286 consensus_queue.cc:237] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } }
W20260619 08:16:54.537961 17140 tablet.cc:2369] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:16:54.538568 16811 catalog_manager.cc:5495] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63357d668da41fe9e08d192624e0932 (127.2.119.195). New cstate: current_term: 1 leader_uuid: "d63357d668da41fe9e08d192624e0932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63357d668da41fe9e08d192624e0932" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 36059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 } health_report { overall_health: UNKNOWN } } }
I20260619 08:16:54.545245 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.542592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.2.119.195:36059)
0619 08:16:54.542669 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:16:54.542671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:16:54.542680 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:16:54.542712 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:16:54.542713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:16:54.542736 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:16:54.542803 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:16:54.542991 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.543597 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.544421 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:16:54.544613 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:16:54.544673 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:16:54.544860 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:16:54.544955 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:16:54.545021 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:16:54.545089 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:16:54.545108 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 08:16:54.547235 17093 raft_consensus.cc:1261] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Refusing update from remote peer d63357d668da41fe9e08d192624e0932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:54.547266 16954 raft_consensus.cc:1261] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Refusing update from remote peer d63357d668da41fe9e08d192624e0932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:16:54.547626 17286 consensus_queue.cc:1035] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09f6a9a368e5409086dc7407f9f2fa17" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:54.547775 17288 consensus_queue.cc:1035] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf678f43c26a41cb8fac51d479cab6a7" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 41215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:16:54.550112 17290 mvcc.cc:205] Tried to move back new op lower bound from 7298486331582246912 to 7298486331546611712. Current Snapshot: MvccSnapshot[applied={T|T < 7298486331582246912}]
I20260619 08:16:54.552395 16812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:54.552476 16812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:16:54.553189 16812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Sending DeleteTablet for 3 replicas of tablet 9096c6c375984c4f95d60d7afa8a1e96
I20260619 08:16:54.553526 17073 tablet_service.cc:1514] Processing DeleteTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:54 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:38956
I20260619 08:16:54.553542 17209 tablet_service.cc:1514] Processing DeleteTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:54 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:59742
I20260619 08:16:54.553648 17146 tablet_replica.cc:330] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: stopping tablet replica
I20260619 08:16:54.553648 16934 tablet_service.cc:1514] Processing DeleteTablet for tablet 9096c6c375984c4f95d60d7afa8a1e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:16:54 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:60300
I20260619 08:16:54.553699 17146 raft_consensus.cc:2229] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:54.553722 17009 tablet_replica.cc:330] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: stopping tablet replica
I20260619 08:16:54.553750 17146 raft_consensus.cc:2258] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:54.553761 17009 raft_consensus.cc:2229] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:16:54.553814 17009 raft_consensus.cc:2258] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:54.554095 17146 ts_tablet_manager.cc:1905] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.554173 17009 ts_tablet_manager.cc:1905] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.555598 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16850
I20260619 08:16:54.555770 17009 ts_tablet_manager.cc:1918] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.555826 17009 log.cc:1196] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/9096c6c375984c4f95d60d7afa8a1e96
I20260619 08:16:54.556077 17009 ts_tablet_manager.cc:1939] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting consensus metadata
I20260619 08:16:54.557678 17146 ts_tablet_manager.cc:1918] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.557744 17146 log.cc:1196] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/9096c6c375984c4f95d60d7afa8a1e96
I20260619 08:16:54.557972 17146 ts_tablet_manager.cc:1939] T 9096c6c375984c4f95d60d7afa8a1e96 P cf678f43c26a41cb8fac51d479cab6a7: Deleting consensus metadata
I20260619 08:16:54.558307 16799 catalog_manager.cc:4841] TS cf678f43c26a41cb8fac51d479cab6a7 (127.2.119.194:41215): tablet 9096c6c375984c4f95d60d7afa8a1e96 (table test-table [id=a8dd7bff412c43d5a34d0676e18a49ea]) successfully deleted
W20260619 08:16:54.561654 16798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:44627 (error 108)
I20260619 08:16:54.561935 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17005
I20260619 08:16:54.562177 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.562042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44627 (local address 127.0.0.1:60310)
0619 08:16:54.562101 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.562137 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:16:54.561975 17300 tablet_replica.cc:330] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: stopping tablet replica
W20260619 08:16:54.562281 16798 catalog_manager.cc:4584] TS 09f6a9a368e5409086dc7407f9f2fa17 (127.2.119.193:44627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9096c6c375984c4f95d60d7afa8a1e96: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44627: connect: Connection refused (error 111)
I20260619 08:16:54.562884 17300 raft_consensus.cc:2229] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:16:54.563033 17300 raft_consensus.cc:2258] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:16:54.563622 17300 ts_tablet_manager.cc:1905] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.565454 17300 ts_tablet_manager.cc:1918] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.565539 17300 log.cc:1196] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/9096c6c375984c4f95d60d7afa8a1e96
I20260619 08:16:54.565812 17300 ts_tablet_manager.cc:1939] T 9096c6c375984c4f95d60d7afa8a1e96 P d63357d668da41fe9e08d192624e0932: Deleting consensus metadata
I20260619 08:16:54.566397 16796 catalog_manager.cc:4841] TS d63357d668da41fe9e08d192624e0932 (127.2.119.195:36059): tablet 9096c6c375984c4f95d60d7afa8a1e96 (table test-table [id=a8dd7bff412c43d5a34d0676e18a49ea]) successfully deleted
I20260619 08:16:54.567446 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17143
I20260619 08:16:54.572587 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 16779
I20260619 08:16:54.577908 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--webserver_interface=127.2.119.254
--webserver_port=40869
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:54.660678 17302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.660907 17302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.660930 17302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.662195 17302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:16:54.662271 17302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.662287 17302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.662300 17302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:16:54.662312 17302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:16:54.663825 17302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.664088 17302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.664342 17302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.669534 17307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.669485 17308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.669652 17310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.670159 17302 server_base.cc:1034] running on GCE node
I20260619 08:16:54.670415 17302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.670689 17302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.671859 17302 hybrid_clock.cc:648] HybridClock initialized: now 1781857014671846 us; error 32 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.674767 17302 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:16:54.675254 17302 webserver.cc:460] Webserver started at http://127.2.119.254:40869/ using document root <none> and password file <none>
I20260619 08:16:54.675494 17302 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.675565 17302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.676975 17302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.677757 17317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.677935 17302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:16:54.678041 17302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ca02f7da742743e5b7d20b4c704e2143"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "abe8b3c8918c8a9d8bcf28ff4c71f2ff"
server_key_iv: "f03b4987ddc438ec98db36328240fea0"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.678416 17302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.693396 17302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.695777 17302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.695955 17302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.700104 17302 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45921
I20260619 08:16:54.700161 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45921 every 8 connection(s)
I20260619 08:16:54.700527 17302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:16:54.702169 17370 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:16:54.702853 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17302
I20260619 08:16:54.702965 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap starting.
I20260619 08:16:54.703161 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:44627
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=42809
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:16:54.712145 17370 log.cc:824] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Log is configured to *not* fsync() on all Append() calls
I20260619 08:16:54.714471 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:16:54.714787 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143: Bootstrap complete.
I20260619 08:16:54.717015 17370 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.717383 17370 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Initialized, Role: FOLLOWER
I20260619 08:16:54.717583 17370 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.717661 17370 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:16:54.717713 17370 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:16:54.717761 17370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:16:54.718746 17370 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.718871 17370 leader_election.cc:302] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca02f7da742743e5b7d20b4c704e2143; no voters:
I20260619 08:16:54.719079 17370 leader_election.cc:288] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:16:54.719113 17374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:16:54.719221 17374 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [term 3 LEADER]: Becoming Leader. State: Replica: ca02f7da742743e5b7d20b4c704e2143, State: Running, Role: LEADER
I20260619 08:16:54.719316 17374 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } }
I20260619 08:16:54.719353 17370 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:16:54.719581 17376 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca02f7da742743e5b7d20b4c704e2143. Latest consensus state: current_term: 3 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:54.719787 17376 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:54.719732 17375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ca02f7da742743e5b7d20b4c704e2143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca02f7da742743e5b7d20b4c704e2143" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45921 } } }
I20260619 08:16:54.720055 17375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca02f7da742743e5b7d20b4c704e2143 [sys.catalog]: This master's current role is: LEADER
I20260619 08:16:54.720191 17382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:16:54.721016 17382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:16:54.721109 17382 catalog_manager.cc:1259] Loaded cluster ID: 0f7bacb7758348458bf51f07d547e6e0
I20260619 08:16:54.721217 17382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 08:16:54.721611 17382 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7c40de29ad google::LogMessage::Fail() at ??:0
@ 0x7f7c40de588a google::LogMessage::SendToLog() at ??:0
@ 0x7f7c40de24de google::LogMessage::Flush() at ??:0
@ 0x7f7c40de3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7c48d030f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f7c48d0c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7c42cb00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7c42ca5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7c43b096db start_thread at ??:0
@ 0x7f7c3f8e061f clone at ??:0
W20260619 08:16:54.805135 17372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.805394 17372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.805423 17372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.806577 17372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.806617 17372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.806643 17372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:16:54.808087 17372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:44627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=42809
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.808323 17372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.808559 17372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.813603 17409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.813618 17407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.813668 17406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.813970 17372 server_base.cc:1034] running on GCE node
I20260619 08:16:54.814157 17372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.814409 17372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.815582 17372 hybrid_clock.cc:648] HybridClock initialized: now 1781857014815563 us; error 32 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.818212 17372 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:16:54.818747 17372 webserver.cc:460] Webserver started at http://127.2.119.193:42809/ using document root <none> and password file <none>
I20260619 08:16:54.818971 17372 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.819041 17372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.820685 17372 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.821482 17416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.821671 17372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.821764 17372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "09f6a9a368e5409086dc7407f9f2fa17"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "30522088741e94c2987605caf123b0f1"
server_key_iv: "fe2f543fd8967092c11603b4e52dbfb2"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.822190 17372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.839558 17372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.841991 17372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.842159 17372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.842434 17372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:54.842882 17423 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260619 08:16:54.844787 17423 ts_tablet_manager.cc:1852] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 08:16:54.845196 17423 ts_tablet_manager.cc:1905] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:16:54.847741 17423 ts_tablet_manager.cc:1918] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:16:54.847868 17423 ts_tablet_manager.cc:1939] T 9096c6c375984c4f95d60d7afa8a1e96 P 09f6a9a368e5409086dc7407f9f2fa17: Deleting consensus metadata
I20260619 08:16:54.848217 17372 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 08:16:54.848282 17372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 08:16:54.848330 17372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:54.848374 17372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.854864 17372 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:44627
I20260619 08:16:54.854955 17530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:44627 every 8 connection(s)
I20260619 08:16:54.855257 17372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:16:54.859215 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17372
I20260619 08:16:54.859503 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:41215
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=35485
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:54.930203 17533 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:16:54.856012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:45873)
0619 08:16:54.856231 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.856271 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:16:54.856781 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:16:54.929957 (+ 73176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
W20260619 08:16:54.930491 17531 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:45921 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:45921: Client connection negotiation failed: client connection to 127.2.119.254:45921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:16:54.930860 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.930739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:54755)
0619 08:16:54.930795 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.930839 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 08:16:54.931228 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:54.931115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:52017)
0619 08:16:54.931166 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:54.931198 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:16:54.931354 17531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:16:54.963748 17534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:54.963950 17534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:54.963974 17534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:54.965176 17534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:54.965220 17534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:54.965247 17534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:16:54.966719 17534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:41215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=35485
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:54.966902 17534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:54.967077 17534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:54.972384 17542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:54.972384 17540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.972608 17534 server_base.cc:1034] running on GCE node
W20260619 08:16:54.972404 17539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:54.972893 17534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:54.973152 17534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:54.974319 17534 hybrid_clock.cc:648] HybridClock initialized: now 1781857014974305 us; error 55 us; skew 500 ppm
Jun 19 08:16:54 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857014, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:54.976984 17534 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:16:54.977517 17534 webserver.cc:460] Webserver started at http://127.2.119.194:35485/ using document root <none> and password file <none>
I20260619 08:16:54.977747 17534 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:54.977816 17534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:54.979193 17534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.979837 17549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.980144 17534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:16:54.980262 17534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "cf678f43c26a41cb8fac51d479cab6a7"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "fae2273df2bf81f7e74fdec26539b66b"
server_key_iv: "945946d9823329afc818a98557896d02"
server_key_version: "encryptionkey@0"
I20260619 08:16:54.980636 17534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:54.991760 17534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:54.994213 17534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:54.994380 17534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:54.994626 17534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:54.995033 17534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:54.995090 17534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:54.995132 17534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:54.995158 17534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:55.000959 17534 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:41215
I20260619 08:16:55.001055 17662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:41215 every 8 connection(s)
I20260619 08:16:55.001400 17534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:16:55.002813 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.002151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:33185)
0619 08:16:55.002386 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.002459 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
W20260619 08:16:55.003067 17663 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:45921 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:45921: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
I20260619 08:16:55.003338 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.003252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:41657)
0619 08:16:55.003291 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.003318 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 08:16:55.003643 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.003589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:55919)
0619 08:16:55.003609 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.003628 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 08:16:55.003773 17663 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:16:55.005470 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17534
I20260619 08:16:55.005714 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:36059
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:45921
--webserver_port=44167
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:16:55.086699 17666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:16:55.086915 17666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:16:55.086939 17666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:16:55.088063 17666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:16:55.088099 17666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:16:55.088125 17666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:16:55.089552 17666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:36059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=44167
--tserver_master_addrs=127.2.119.254:45921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:16:55.089744 17666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:16:55.089913 17666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:16:55.095124 17671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:16:55.095088 17672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:55.095283 17666 server_base.cc:1034] running on GCE node
W20260619 08:16:55.095124 17674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:16:55.095593 17666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:16:55.095855 17666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:16:55.096997 17666 hybrid_clock.cc:648] HybridClock initialized: now 1781857015096979 us; error 32 us; skew 500 ppm
Jun 19 08:16:55 dist-test-slave-d0b5 krb5kdc[16233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857015, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:16:55.099558 17666 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:16:55.100065 17666 webserver.cc:460] Webserver started at http://127.2.119.195:44167/ using document root <none> and password file <none>
I20260619 08:16:55.100337 17666 fs_manager.cc:359] Metadata directory not provided
I20260619 08:16:55.100401 17666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:16:55.101790 17666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:16:55.102459 17681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:55.102644 17666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:16:55.102735 17666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "d63357d668da41fe9e08d192624e0932"
format_stamp: "Formatted at 2026-06-19 08:16:53 on dist-test-slave-d0b5"
server_key: "49d5b16a4c63c331dd77208e1ac60cc2"
server_key_iv: "27fb6149d380143a8eb75fb0c13606a0"
server_key_version: "encryptionkey@0"
I20260619 08:16:55.103066 17666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:16:55.132992 17666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:16:55.135211 17666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:16:55.135366 17666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:16:55.135627 17666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:16:55.135979 17666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:16:55.136042 17666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:55.136078 17666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:16:55.136106 17666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:16:55.142136 17666 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:36059
I20260619 08:16:55.142182 17794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:36059 every 8 connection(s)
I20260619 08:16:55.142530 17666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:16:55.143647 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.143184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:55957)
0619 08:16:55.143395 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.143489 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
W20260619 08:16:55.144127 17795 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:45921 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:45921: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
I20260619 08:16:55.144440 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.144335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:40555)
0619 08:16:55.144387 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.144422 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:16:55.144776 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.144684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:49423)
0619 08:16:55.144728 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.144756 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:16:55.144935 17795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:16:55.151081 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17666
I20260619 08:16:55.932135 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:55.931883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:50623)
0619 08:16:55.932023 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:55.932068 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260619 08:16:56.004407 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:56.004195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:53189)
0619 08:16:56.004318 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:56.004358 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260619 08:16:56.145674 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:56.145409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:57333)
0619 08:16:56.145580 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:56.145617 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":75,"threads_started":1}
I20260619 08:16:56.932909 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:56.932692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:34823)
0619 08:16:56.932822 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:56.932862 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260619 08:16:57.005054 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:57.004867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:48495)
0619 08:16:57.004980 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:57.005012 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260619 08:16:57.146412 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:57.146167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:41793)
0619 08:16:57.146307 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:57.146349 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260619 08:16:57.933624 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:57.933410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:54645)
0619 08:16:57.933538 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:57.933575 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260619 08:16:58.005769 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:58.005549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:54677)
0619 08:16:58.005683 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:58.005725 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260619 08:16:58.147244 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:58.147022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:37627)
0619 08:16:58.147160 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:58.147199 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260619 08:16:58.934433 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:58.934193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:36979)
0619 08:16:58.934329 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:58.934373 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260619 08:16:59.006631 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:59.006346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:35941)
0619 08:16:59.006520 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:59.006574 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260619 08:16:59.148087 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:59.147763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:56745)
0619 08:16:59.147988 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:59.148023 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":141,"threads_started":1}
I20260619 08:16:59.935173 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:16:59.934951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:57003)
0619 08:16:59.935086 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:16:59.935127 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260619 08:17:00.007477 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:00.007193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:60801)
0619 08:17:00.007375 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:00.007426 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260619 08:17:00.148793 17813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:00.148586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:33771)
0619 08:17:00.148713 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:00.148749 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260619 08:17:00.935971 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:00.935714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:54215)
0619 08:17:00.935866 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:00.935922 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260619 08:17:01.008249 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:01.007987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:55563)
0619 08:17:01.008165 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:01.008204 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260619 08:17:01.149565 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:01.149327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:60007)
0619 08:17:01.149470 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:01.149520 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260619 08:17:01.936803 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:01.936562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:57273)
0619 08:17:01.936712 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:01.936755 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260619 08:17:02.009028 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:02.008807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:34029)
0619 08:17:02.008934 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:02.008979 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260619 08:17:02.150389 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:02.150110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:44295)
0619 08:17:02.150286 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:02.150338 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260619 08:17:02.937680 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:02.937429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:38215)
0619 08:17:02.937585 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:02.937631 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260619 08:17:03.009773 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:03.009538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:33691)
0619 08:17:03.009690 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:03.009726 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260619 08:17:03.151163 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:03.150921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:39689)
0619 08:17:03.151076 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:03.151115 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260619 08:17:03.938709 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:03.938407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:40833)
0619 08:17:03.938617 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:03.938662 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":91,"threads_started":1}
I20260619 08:17:04.010576 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:04.010329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:58715)
0619 08:17:04.010478 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:04.010528 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260619 08:17:04.152093 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:04.151847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:60893)
0619 08:17:04.151976 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:04.152015 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260619 08:17:04.939529 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:04.939296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:58837)
0619 08:17:04.939441 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:04.939480 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260619 08:17:05.011310 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:05.011085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:58715)
0619 08:17:05.011222 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:05.011260 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260619 08:17:05.152838 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:05.152620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:49819)
0619 08:17:05.152753 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:05.152793 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260619 08:17:05.940552 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:05.940218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:56871)
0619 08:17:05.940409 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:05.940457 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":140,"threads_started":1}
I20260619 08:17:06.012089 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:06.011838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:57689)
0619 08:17:06.011991 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:06.012039 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":61,"threads_started":1}
I20260619 08:17:06.153719 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:06.153471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:54007)
0619 08:17:06.153631 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:06.153670 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260619 08:17:06.941365 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:06.941112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:55043)
0619 08:17:06.941253 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:06.941317 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260619 08:17:07.012863 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:07.012600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:55053)
0619 08:17:07.012771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:07.012811 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260619 08:17:07.154606 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:07.154334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:36845)
0619 08:17:07.154515 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:07.154558 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":81,"threads_started":1}
I20260619 08:17:07.942390 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:07.942046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:51911)
0619 08:17:07.942232 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:07.942317 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":66,"threads_started":1}
I20260619 08:17:08.013715 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:08.013478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:54865)
0619 08:17:08.013623 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:08.013664 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260619 08:17:08.155386 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:08.155117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:44063)
0619 08:17:08.155266 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:08.155306 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260619 08:17:08.943204 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:08.942891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:33669)
0619 08:17:08.943118 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:08.943157 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":66,"threads_started":1}
I20260619 08:17:09.014557 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:09.014252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:46097)
0619 08:17:09.014463 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:09.014510 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":60,"threads_started":1}
I20260619 08:17:09.156126 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:09.155860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:37727)
0619 08:17:09.156042 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:09.156079 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260619 08:17:09.944155 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:09.943918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.193:43459)
0619 08:17:09.944062 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:09.944106 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260619 08:17:10.015512 17842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:10.015247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.194:46217)
0619 08:17:10.015415 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:10.015459 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260619 08:17:10.156320 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17372
I20260619 08:17:10.157596 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:10.157073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45921 (local address 127.2.119.195:58557)
0619 08:17:10.157514 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:10.157555 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":54,"threads_started":1}
I20260619 08:17:10.161115 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17534
I20260619 08:17:10.165548 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17666
2026-06-19T08:17:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18185 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:17:10 dist-test-slave-d0b5 krb5kdc[17846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:17:10 dist-test-slave-d0b5 krb5kdc[17846](info): set up 2 sockets
Jun 19 08:17:10 dist-test-slave-d0b5 krb5kdc[17846](info): commencing operation
krb5kdc: starting...
W20260619 08:17:11.218344 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:17:11.336289 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:11.416785 17859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:11.416991 17859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:11.417012 17859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:11.418107 17859 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:17:11.418150 17859 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:17:11.418164 17859 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:17:11.418176 17859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:17:11.418190 17859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:11.418202 17859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:11.418215 17859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:17:11.418226 17859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:17:11.419617 17859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:11.419808 17859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:11.419971 17859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:11.425423 17864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.425451 17859 server_base.cc:1034] running on GCE node
W20260619 08:17:11.425427 17867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:11.425410 17865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.425944 17859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:11.426206 17859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:11.427357 17859 hybrid_clock.cc:648] HybridClock initialized: now 1781857031427345 us; error 32 us; skew 500 ppm
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:11.429975 17859 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:17:11.430536 17859 webserver.cc:460] Webserver started at http://127.2.119.254:33043/ using document root <none> and password file <none>
I20260619 08:17:11.430783 17859 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:11.430830 17859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:11.430953 17859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:17:11.432050 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "766543abf11b4ed1865250709dbaeb6f"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "446b72067aa86f2fc3fe526e88c07f7b"
server_key_iv: "f0ec8c812d2184ecf55aeaa23377c82a"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.432415 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "766543abf11b4ed1865250709dbaeb6f"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "446b72067aa86f2fc3fe526e88c07f7b"
server_key_iv: "f0ec8c812d2184ecf55aeaa23377c82a"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.433835 17859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:17:11.434624 17874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.434875 17859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:11.434968 17859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "766543abf11b4ed1865250709dbaeb6f"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "446b72067aa86f2fc3fe526e88c07f7b"
server_key_iv: "f0ec8c812d2184ecf55aeaa23377c82a"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.435042 17859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:11.456324 17859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:11.458681 17859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:11.458838 17859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:11.463007 17859 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:38433
I20260619 08:17:11.463042 17926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:38433 every 8 connection(s)
I20260619 08:17:11.463403 17859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:17:11.464295 17927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:11.466630 17927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap starting.
I20260619 08:17:11.467377 17927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:11.467762 17927 log.cc:824] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:11.468396 17927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: No bootstrap required, opened a new log
I20260619 08:17:11.470424 17927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:11.470562 17927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:11.470583 17927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Initialized, Role: FOLLOWER
I20260619 08:17:11.470692 17927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:11.470770 17927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:17:11.470808 17927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:17:11.470858 17927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:11.471203 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17859
I20260619 08:17:11.471298 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:17:11.471480 17927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:11.471566 2527 external_mini_cluster.cc:1428] Setting key 6e41582c50824505e9d47844a2ea5551
I20260619 08:17:11.471593 17927 leader_election.cc:302] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766543abf11b4ed1865250709dbaeb6f; no voters:
I20260619 08:17:11.471788 17927 leader_election.cc:288] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:17:11.471824 17930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:17:11.471938 17930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 LEADER]: Becoming Leader. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Running, Role: LEADER
I20260619 08:17:11.472054 17927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:17:11.472074 17930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:11.472944 17931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:11.472957 17932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766543abf11b4ed1865250709dbaeb6f. Latest consensus state: current_term: 1 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:11.473035 17931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:11.473070 17932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:11.473371 17941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:17:11.473989 17941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:17:11.475661 17941 catalog_manager.cc:1347] Generated new cluster ID: 27ddbff3008546f7b53fa6a510b246cb
I20260619 08:17:11.475718 17941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:11.483826 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.472261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.2.119.254:38433)
0619 08:17:11.472411 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:11.472421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:11.472594 (+ 173us) server_negotiation.cc:407] Connection header received
0619 08:17:11.472744 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:11.472753 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:11.472805 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:11.472939 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:11.473920 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.475236 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.476317 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.476951 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.477042 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.480014 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:11.480017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:11.480018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:11.480030 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:11.481961 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.482419 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.482422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.482423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.482540 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.482763 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.482764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.482764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.483063 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:11.483180 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:11.483348 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 08:17:11.483592 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260619 08:17:11.508754 17941 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:17:11.509492 17941 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:17:11.516829 17941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Generated new TSK 0
I20260619 08:17:11.517062 17941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:17:11.552597 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:11.634548 17953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:11.634755 17953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:11.634780 17953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:11.635861 17953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:11.635896 17953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:11.635921 17953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:17:11.637427 17953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:11.637611 17953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:11.637799 17953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:11.643002 17959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:11.642988 17961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.643092 17953 server_base.cc:1034] running on GCE node
W20260619 08:17:11.642989 17958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.643488 17953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:11.643759 17953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:11.644912 17953 hybrid_clock.cc:648] HybridClock initialized: now 1781857031644892 us; error 42 us; skew 500 ppm
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:11.647620 17953 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:17:11.648033 17953 webserver.cc:460] Webserver started at http://127.2.119.193:38157/ using document root <none> and password file <none>
I20260619 08:17:11.648290 17953 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:11.648365 17953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:11.648517 17953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:17:11.649633 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "54bb4fb8edfc72f27d8ea6eb7a55f378"
server_key_iv: "9becf04fac572391b7ffbdc93b09fe2d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.650002 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "54bb4fb8edfc72f27d8ea6eb7a55f378"
server_key_iv: "9becf04fac572391b7ffbdc93b09fe2d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.651701 17953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:17:11.652594 17968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.652810 17953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:11.652925 17953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "54bb4fb8edfc72f27d8ea6eb7a55f378"
server_key_iv: "9becf04fac572391b7ffbdc93b09fe2d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.653074 17953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:11.662963 17953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:11.665251 17953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:11.665468 17953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:11.665693 17953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:11.665993 17953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:17:11.666057 17953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.666100 17953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:17:11.666128 17953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.671852 17953 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34571
I20260619 08:17:11.671922 18081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34571 every 8 connection(s)
I20260619 08:17:11.672241 17953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:17:11.677791 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 17953
I20260619 08:17:11.677909 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:17:11.678205 2527 external_mini_cluster.cc:1428] Setting key 7e916592c7d658d857a48cc1507fd952
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:11.682619 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.673043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:52815 (local address 127.2.119.254:38433)
0619 08:17:11.673123 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:11.673125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:11.673351 (+ 226us) server_negotiation.cc:407] Connection header received
0619 08:17:11.673854 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:11.673855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:11.673871 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:11.673972 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:11.674985 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.675568 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.676613 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.676804 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.676859 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.679664 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:11.679665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:11.679665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:11.679673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:11.681115 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.681618 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.681620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.681620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.681689 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.681965 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.681966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.681966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.682083 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:11.682155 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:11.682437 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 08:17:11.682520 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:17:11.682798 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.673054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:52815)
0619 08:17:11.673203 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:11.673244 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:11.673752 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:11.674107 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:11.674112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:11.674686 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:11.674884 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.674889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.675678 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.675679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.676532 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.676535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.676914 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.676915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.677090 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.677733 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:11.677744 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:11.679554 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:11.681219 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.681226 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.681228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.681527 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.681767 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.681768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.681769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.681891 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.682157 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:11.682164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:11.682303 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 08:17:11.682403 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 08:17:11.682560 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260619 08:17:11.683290 18082 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:11.683373 18082 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:11.683580 18082 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:11.684136 17891 ts_manager.cc:194] Registered new tserver with Master: f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571)
I20260619 08:17:11.684748 17891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:52815
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:17:11.704559 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:11.785528 18087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:11.785738 18087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:11.785763 18087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:11.786863 18087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:11.786903 18087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:11.786929 18087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:17:11.788313 18087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:11.788530 18087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:11.788793 18087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:11.794412 18092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:11.794394 18093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.794523 18087 server_base.cc:1034] running on GCE node
W20260619 08:17:11.794397 18095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.794847 18087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:11.795102 18087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:11.796294 18087 hybrid_clock.cc:648] HybridClock initialized: now 1781857031796264 us; error 46 us; skew 500 ppm
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:11.798976 18087 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:17:11.799472 18087 webserver.cc:460] Webserver started at http://127.2.119.194:43989/ using document root <none> and password file <none>
I20260619 08:17:11.799721 18087 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:11.799793 18087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:11.799940 18087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:17:11.800956 18087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "1a904c89de62a538739d4bbcd0e5e337"
server_key_iv: "67b2413ec5c409309057087ecddd282d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.801378 18087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "1a904c89de62a538739d4bbcd0e5e337"
server_key_iv: "67b2413ec5c409309057087ecddd282d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.802698 18087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:11.803447 18102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.803675 18087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:11.803774 18087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "1a904c89de62a538739d4bbcd0e5e337"
server_key_iv: "67b2413ec5c409309057087ecddd282d"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.803855 18087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:11.819298 18087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:11.821259 18087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:11.821441 18087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:11.821667 18087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:11.821976 18087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:17:11.822031 18087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.822073 18087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:17:11.822101 18087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.827875 18087 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:35491
I20260619 08:17:11.827919 18215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:35491 every 8 connection(s)
I20260619 08:17:11.828284 18087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:17:11.829355 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18087
I20260619 08:17:11.829478 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:17:11.829666 2527 external_mini_cluster.cc:1428] Setting key 30ba66a3f4488f1259b76196facfc91d
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:17:11.837888 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.829023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:57537 (local address 127.2.119.254:38433)
0619 08:17:11.829102 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:11.829104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:11.829319 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:17:11.829949 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:11.829951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:11.829963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:11.830053 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:11.831073 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.831579 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.832568 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.832770 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.832855 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.835031 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:11.835032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:11.835032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:11.835040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:11.836439 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.836864 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.836866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.836866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.836925 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.837187 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.837188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.837188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.837347 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:11.837433 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:11.837715 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 08:17:11.837796 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 08:17:11.838073 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.828965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:57537)
0619 08:17:11.829172 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:11.829207 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:11.829844 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:11.830139 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:11.830143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:11.830782 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:11.830971 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.830976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.831677 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.831678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.832466 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.832467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.832866 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.832867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.832991 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.833550 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:11.833557 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:11.834935 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:11.836536 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.836543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.836546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.836773 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.836995 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.836996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.836996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.837105 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.837425 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:11.837431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:11.837569 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:17:11.837683 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 08:17:11.837839 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
I20260619 08:17:11.838671 18216 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:11.838786 18216 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:11.838964 18216 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:11.839377 17891 ts_manager.cc:194] Registered new tserver with Master: 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
I20260619 08:17:11.839830 17891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:57537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:17:11.857214 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:11.938218 18221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:11.938408 18221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:11.938431 18221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:11.939505 18221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:11.939539 18221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:11.939565 18221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:17:11.940922 18221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:11.941103 18221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:11.941274 18221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:17:11.946368 18221 server_base.cc:1034] running on GCE node
W20260619 08:17:11.946357 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:11.946342 18229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:11.946352 18226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:11.946841 18221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:11.947048 18221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:11.948199 18221 hybrid_clock.cc:648] HybridClock initialized: now 1781857031948178 us; error 40 us; skew 500 ppm
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:11.950842 18221 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:17:11.951309 18221 webserver.cc:460] Webserver started at http://127.2.119.195:34823/ using document root <none> and password file <none>
I20260619 08:17:11.951593 18221 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:11.951638 18221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:11.951795 18221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:17:11.952816 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "915d4aa7921c48d5911715fc6c8c6686"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "2f63f6028645888a032720f8a240a070"
server_key_iv: "209d908a63075d4ec85bdd573b726865"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.953189 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "915d4aa7921c48d5911715fc6c8c6686"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "2f63f6028645888a032720f8a240a070"
server_key_iv: "209d908a63075d4ec85bdd573b726865"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.954497 18221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:11.955317 18236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.955513 18221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:11.955592 18221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "915d4aa7921c48d5911715fc6c8c6686"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "2f63f6028645888a032720f8a240a070"
server_key_iv: "209d908a63075d4ec85bdd573b726865"
server_key_version: "encryptionkey@0"
I20260619 08:17:11.955650 18221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:11.966234 18221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:11.968003 18221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:11.968111 18221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:11.968317 18221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:11.968596 18221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:17:11.968628 18221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.968658 18221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:17:11.968672 18221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:11.974426 18221 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33755
I20260619 08:17:11.974498 18349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33755 every 8 connection(s)
I20260619 08:17:11.974802 18221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:17:11 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857031, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:11.981971 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18221
I20260619 08:17:11.982076 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:17:11.982301 2527 external_mini_cluster.cc:1428] Setting key 0549dc28ac6fa2a0290d0ad2886a8a5a
I20260619 08:17:11.984117 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.975511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:47455 (local address 127.2.119.254:38433)
0619 08:17:11.975593 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:11.975595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:11.975775 (+ 180us) server_negotiation.cc:407] Connection header received
0619 08:17:11.976270 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:11.976271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:11.976281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:11.976344 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:11.977243 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.977997 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.978951 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.979131 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.979194 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.981172 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:11.981173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:11.981173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:11.981179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:11.982663 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.983124 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.983125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.983125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.983218 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:11.983493 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:11.983494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:11.983495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:11.983604 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:11.983683 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:11.983950 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 08:17:11.984025 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:17:11.984311 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.975478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:47455)
0619 08:17:11.975628 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:11.975674 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:11.976143 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:11.976397 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:11.976400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:11.976962 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:11.977142 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.977147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.978074 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.978075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.978870 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:11.978872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.979213 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.979213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:11.979306 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.979797 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:11.979804 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:11.981085 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:11.982749 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.982756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.982758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.983058 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.983303 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:11.983304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:11.983304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:11.983416 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:11.983675 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:11.983680 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:11.983816 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 08:17:11.983928 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:17:11.984082 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260619 08:17:11.984766 18350 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:11.984858 18350 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:11.984992 18350 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:11.985378 17891 ts_manager.cc:194] Registered new tserver with Master: 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755)
I20260619 08:17:11.985764 17891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:47455
I20260619 08:17:11.994014 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:17:12.001204 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:11.995371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50436 (local address 127.2.119.254:38433)
0619 08:17:11.995421 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:11.995423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:11.995435 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:17:11.995517 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:11.995518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:11.995527 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:11.995575 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:11.996133 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.996629 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.997424 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:11.997613 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:11.997698 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:11.998839 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:11.998841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:11.998841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:11.998848 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:12.000042 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:12.000386 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:12.000388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:12.000388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:12.000445 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:12.000639 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:12.000639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:12.000640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:12.000786 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:12.000892 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.000990 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.001065 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:17:12.002947 17891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:17:12.004478 17891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:17:12.010931 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.2.119.194:35491)
0619 08:17:12.007116 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.007121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.007133 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:17:12.007173 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.007175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.007190 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.007303 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:12.007424 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008526 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.009774 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.010454 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.010542 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.010601 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.010806 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.010846 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260619 08:17:12.010993 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.0.0.1:54156)
0619 08:17:12.006800 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.006826 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.006878 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:12.007304 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:12.007305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:12.007307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:12.007367 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.007368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008633 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.008634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.009716 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.009718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.010583 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.010583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.010675 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.010691 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:17:12.010783 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:17:12.010878 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":6,"spinlock_wait_cycles":11648,"thread_start_us":75,"threads_started":1}
I20260619 08:17:12.011157 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.2.119.193:34571)
0619 08:17:12.007157 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.007162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.007313 (+ 151us) server_negotiation.cc:407] Connection header received
0619 08:17:12.007356 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.007358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.007373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.007437 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:12.007630 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008648 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.009718 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.010455 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.010542 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.010601 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.011041 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.011083 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260619 08:17:12.011473 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.0.0.1:40952)
0619 08:17:12.007187 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.007198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.007254 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:12.007514 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:12.007514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:12.007516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:12.007555 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.007557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008751 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.008752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.009621 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.009623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.010581 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.010583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.010675 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.010691 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:17:12.010783 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:17:12.010878 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":44,"thread_start_us":63,"threads_started":1}
I20260619 08:17:12.011714 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.0.0.1:40958)
0619 08:17:12.006782 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.006818 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.006877 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:12.007223 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:12.007226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:12.007229 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:12.007288 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.007290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008343 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.008344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.009622 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.009625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.011447 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.011449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.011527 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.011531 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:17:12.011595 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:17:12.011643 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":5,"thread_start_us":63,"threads_started":1}
I20260619 08:17:12.011830 18150 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4e8348427a485abf57a5bcaef33124 (DEFAULT_TABLE table=test-table [id=d32c5d527a63404bbd045225665c4b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:12.011967 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.006690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.2.119.195:33755)
0619 08:17:12.007018 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.007023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.007033 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:17:12.007070 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.007072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.007089 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.007212 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:12.007371 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.008240 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.010638 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.011352 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.011446 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.011497 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.011823 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.011872 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":74,"threads_started":1}
I20260619 08:17:12.012197 18150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4e8348427a485abf57a5bcaef33124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:12.012542 18284 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4e8348427a485abf57a5bcaef33124 (DEFAULT_TABLE table=test-table [id=d32c5d527a63404bbd045225665c4b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:12.012836 18284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4e8348427a485abf57a5bcaef33124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:12.013726 18016 tablet_service.cc:1467] Processing CreateTablet for tablet 4a4e8348427a485abf57a5bcaef33124 (DEFAULT_TABLE table=test-table [id=d32c5d527a63404bbd045225665c4b20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:12.014040 18016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a4e8348427a485abf57a5bcaef33124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:12.015588 18365 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap starting.
I20260619 08:17:12.015811 18364 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap starting.
I20260619 08:17:12.016387 18364 tablet_bootstrap.cc:654] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:12.016916 18364 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:12.016957 18366 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap starting.
I20260619 08:17:12.017637 18366 tablet_bootstrap.cc:654] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:12.017763 18364 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: No bootstrap required, opened a new log
I20260619 08:17:12.017773 18365 tablet_bootstrap.cc:654] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:12.017832 18364 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:17:12.018260 18366 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:12.018364 18365 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:12.018977 18366 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: No bootstrap required, opened a new log
I20260619 08:17:12.019044 18366 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:17:12.019201 18365 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: No bootstrap required, opened a new log
I20260619 08:17:12.019301 18365 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 08:17:12.019810 18364 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.019994 18364 raft_consensus.cc:374] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:12.020038 18364 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c39b3934c1f4d3dbf178dff3039ce0a, State: Initialized, Role: FOLLOWER
I20260619 08:17:12.020157 18364 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.020453 18364 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:17:12.020488 18216 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
I20260619 08:17:12.020485 18366 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.020654 18366 raft_consensus.cc:374] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:12.020707 18366 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Initialized, Role: FOLLOWER
I20260619 08:17:12.020845 18366 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.021180 18366 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:17:12.021378 18082 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
I20260619 08:17:12.021924 18365 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.022076 18365 raft_consensus.cc:374] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:12.022130 18365 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915d4aa7921c48d5911715fc6c8c6686, State: Initialized, Role: FOLLOWER
I20260619 08:17:12.022248 18365 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.022523 18365 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:17:12.022527 18350 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
I20260619 08:17:12.074179 18370 raft_consensus.cc:482] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:17:12.074306 18370 raft_consensus.cc:504] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.074680 18370 leader_election.cc:288] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571), 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755)
I20260619 08:17:12.078713 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.074944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:33075 (local address 127.2.119.195:33755)
0619 08:17:12.075001 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.075003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.075187 (+ 184us) server_negotiation.cc:407] Connection header received
0619 08:17:12.075235 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.075236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.075246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.075297 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:12.075445 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.076116 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.077699 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.078295 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.078374 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.078383 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.078599 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.078640 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:17:12.078720 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.074831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.2.119.194:51023)
0619 08:17:12.074885 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.074897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.074938 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:12.075077 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:12.075078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:12.075080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:12.075158 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.075160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.076142 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.076142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.077589 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.077591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.078412 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.078413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.078507 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.078509 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:17:12.078585 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:17:12.078629 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":78}
I20260619 08:17:12.078728 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.074867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.2.119.194:33075)
0619 08:17:12.075061 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.075070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.075118 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:12.075321 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:12.075322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:12.075323 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:12.075374 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.075375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.076194 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.076195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.077584 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:12.077588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.078412 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.078413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:12.078503 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.078507 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:17:12.078595 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:17:12.078628 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":29,"thread_start_us":119,"threads_started":1}
W20260619 08:17:12.078835 18217 tablet.cc:2369] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:17:12.079061 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.074893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:51023 (local address 127.2.119.193:34571)
0619 08:17:12.074951 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.074953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.074962 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:17:12.074995 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.074996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.075005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.075068 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:12.075248 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.076045 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.077700 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.078295 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.078375 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.078383 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.078599 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.078640 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:17:12.079429 18036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4e8348427a485abf57a5bcaef33124" candidate_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" is_pre_election: true
I20260619 08:17:12.079432 18304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4e8348427a485abf57a5bcaef33124" candidate_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915d4aa7921c48d5911715fc6c8c6686" is_pre_election: true
I20260619 08:17:12.079586 18304 raft_consensus.cc:2454] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c39b3934c1f4d3dbf178dff3039ce0a in term 0.
I20260619 08:17:12.079586 18036 raft_consensus.cc:2454] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c39b3934c1f4d3dbf178dff3039ce0a in term 0.
I20260619 08:17:12.079813 18106 leader_election.cc:302] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c39b3934c1f4d3dbf178dff3039ce0a, f3f8b95bef3c4b10881002c4e3d489a2; no voters:
I20260619 08:17:12.079969 18370 raft_consensus.cc:2781] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:17:12.080061 18370 raft_consensus.cc:482] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:17:12.080098 18370 raft_consensus.cc:3037] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:12.080848 18370 raft_consensus.cc:504] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.081002 18370 leader_election.cc:288] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [CANDIDATE]: Term 1 election: Requested vote from peers f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571), 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755)
I20260619 08:17:12.081238 18036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4e8348427a485abf57a5bcaef33124" candidate_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
I20260619 08:17:12.081247 18304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a4e8348427a485abf57a5bcaef33124" candidate_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915d4aa7921c48d5911715fc6c8c6686"
I20260619 08:17:12.081349 18036 raft_consensus.cc:3037] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:12.081387 18304 raft_consensus.cc:3037] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:12.082229 18304 raft_consensus.cc:2454] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c39b3934c1f4d3dbf178dff3039ce0a in term 1.
I20260619 08:17:12.082324 18036 raft_consensus.cc:2454] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c39b3934c1f4d3dbf178dff3039ce0a in term 1.
I20260619 08:17:12.082365 18103 leader_election.cc:302] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c39b3934c1f4d3dbf178dff3039ce0a, 915d4aa7921c48d5911715fc6c8c6686; no voters:
I20260619 08:17:12.082468 18370 raft_consensus.cc:2781] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:17:12.082614 18370 raft_consensus.cc:686] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 LEADER]: Becoming Leader. State: Replica: 5c39b3934c1f4d3dbf178dff3039ce0a, State: Running, Role: LEADER
I20260619 08:17:12.082695 18370 consensus_queue.cc:237] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:12.083509 17891 catalog_manager.cc:5495] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194). New cstate: current_term: 1 leader_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } health_report { overall_health: UNKNOWN } } }
I20260619 08:17:12.106106 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.103617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.2.119.194:35491)
0619 08:17:12.103705 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:12.103709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:12.103722 (+ 13us) server_negotiation.cc:407] Connection header received
0619 08:17:12.103756 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:12.103757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:12.103773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:12.103831 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:17:12.104027 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.104528 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.105269 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:12.105510 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:12.105562 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:12.105703 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:17:12.105839 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:17:12.105901 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:12.105961 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:17:12.105984 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 08:17:12.107882 18036 raft_consensus.cc:1261] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Refusing update from remote peer 5c39b3934c1f4d3dbf178dff3039ce0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:17:12.107884 18304 raft_consensus.cc:1261] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Refusing update from remote peer 5c39b3934c1f4d3dbf178dff3039ce0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:17:12.108225 18374 consensus_queue.cc:1035] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:17:12.108534 18370 consensus_queue.cc:1035] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:17:12.113233 17891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:17:12.113354 17891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:17:12.114262 17891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Sending DeleteTablet for 3 replicas of tablet 4a4e8348427a485abf57a5bcaef33124
I20260619 08:17:12.114698 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17953
I20260619 08:17:12.114743 18016 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:40952
I20260619 08:17:12.114770 18284 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:40958
I20260619 08:17:12.116925 18150 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:54156
I20260619 08:17:12.117182 18388 tablet_replica.cc:330] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: stopping tablet replica
I20260619 08:17:12.117314 18388 raft_consensus.cc:2229] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:17:12.117413 18388 raft_consensus.cc:2258] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 08:17:12.120002 17878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:34571 (error 108)
W20260619 08:17:12.120596 18360 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:17:12.120415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.0.0.1:40954)
0619 08:17:12.120482 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.120496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:12.120545 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34571: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:17:12.120803 18388 ts_tablet_manager.cc:1905] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:17:12.120833 17878 catalog_manager.cc:4584] TS f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a4e8348427a485abf57a5bcaef33124: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34571: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260619 08:17:12.120990 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18087
I20260619 08:17:12.127785 18387 tablet_replica.cc:330] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: stopping tablet replica
I20260619 08:17:12.127935 18387 raft_consensus.cc:2229] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:12.128073 18387 raft_consensus.cc:2258] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:12.128355 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.128202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.0.0.1:54176)
0619 08:17:12.128267 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.128314 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:35491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 08:17:12.128379 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18221
I20260619 08:17:12.128602 18387 ts_tablet_manager.cc:1905] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:12.132329 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 17859
I20260619 08:17:12.132450 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:12.132323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.0.0.1:40962)
0619 08:17:12.132378 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:12.132412 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.195:33755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:17:32.137643 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--webserver_interface=127.2.119.254
--webserver_port=33043
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:32.224210 18390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:32.224418 18390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:32.224440 18390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:32.225664 18390 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:17:32.225709 18390 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:17:32.225723 18390 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:17:32.225734 18390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:17:32.225749 18390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:32.225761 18390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:32.225772 18390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:17:32.225785 18390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:17:32.227341 18390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=33043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:32.227545 18390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:32.227721 18390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:32.237360 18399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.237327 18397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.237396 18396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:32.237447 18390 server_base.cc:1034] running on GCE node
I20260619 08:17:32.237900 18390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:32.238261 18390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:32.239463 18390 hybrid_clock.cc:648] HybridClock initialized: now 1781857052239447 us; error 59 us; skew 500 ppm
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:32.242157 18390 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:17:32.242671 18390 webserver.cc:460] Webserver started at http://127.2.119.254:33043/ using document root <none> and password file <none>
I20260619 08:17:32.242931 18390 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:32.243031 18390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:32.244550 18390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:17:32.245370 18406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:32.245704 18390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:32.245819 18390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "766543abf11b4ed1865250709dbaeb6f"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "446b72067aa86f2fc3fe526e88c07f7b"
server_key_iv: "f0ec8c812d2184ecf55aeaa23377c82a"
server_key_version: "encryptionkey@0"
I20260619 08:17:32.246277 18390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:32.260636 18390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:32.262698 18390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:32.262861 18390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:32.267117 18390 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:38433
I20260619 08:17:32.267153 18458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:38433 every 8 connection(s)
I20260619 08:17:32.267561 18390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:17:32.269176 18459 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:17:32.269928 18459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap starting.
I20260619 08:17:32.273193 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18390
I20260619 08:17:32.273607 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:34571
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=38157
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:17:32.279036 18459 log.cc:824] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:32.281020 18459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:32.281421 18459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap complete.
I20260619 08:17:32.283732 18459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:32.283962 18459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.284152 18459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:32.284219 18459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:17:32.284288 18459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:17:32.284343 18459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:17:32.285478 18459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:32.285593 18459 leader_election.cc:302] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766543abf11b4ed1865250709dbaeb6f; no voters:
I20260619 08:17:32.285799 18459 leader_election.cc:288] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 08:17:32.285836 18463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:17:32.285946 18463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 LEADER]: Becoming Leader. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Running, Role: LEADER
I20260619 08:17:32.286085 18459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:17:32.286099 18463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:32.286398 18463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766543abf11b4ed1865250709dbaeb6f. Latest consensus state: current_term: 2 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:32.286403 18464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:32.286608 18464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:32.286701 18463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:32.287528 18473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:17:32.288331 18473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:17:32.288481 18473 catalog_manager.cc:1259] Loaded cluster ID: 27ddbff3008546f7b53fa6a510b246cb
I20260619 08:17:32.288723 18473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:17:32.289624 18473 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:17:32.289820 18473 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Loaded TSK: 0
I20260619 08:17:32.297605 18473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Generated new TSK 1
I20260619 08:17:32.297729 18473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:17:32.372738 18461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:32.372948 18461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:32.372973 18461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:32.374264 18461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:32.374307 18461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:32.374334 18461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:17:32.375771 18461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=38157
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:32.376029 18461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:32.376277 18461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:32.382110 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.382282 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.382115 18486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:32.382761 18461 server_base.cc:1034] running on GCE node
I20260619 08:17:32.382983 18461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:32.383270 18461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:32.384440 18461 hybrid_clock.cc:648] HybridClock initialized: now 1781857052384428 us; error 32 us; skew 500 ppm
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:32.387127 18461 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:17:32.387550 18461 webserver.cc:460] Webserver started at http://127.2.119.193:38157/ using document root <none> and password file <none>
I20260619 08:17:32.387780 18461 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:32.387931 18461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:32.389457 18461 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:32.390218 18496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:32.390394 18461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:32.390488 18461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "54bb4fb8edfc72f27d8ea6eb7a55f378"
server_key_iv: "9becf04fac572391b7ffbdc93b09fe2d"
server_key_version: "encryptionkey@0"
I20260619 08:17:32.390838 18461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:32.403555 18461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:32.405958 18461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:32.406122 18461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:32.406358 18461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:32.406773 18503 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:17:32.408721 18461 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:17:32.408792 18461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:17:32.408833 18461 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:17:32.409512 18461 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:17:32.409562 18461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:32.409643 18503 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap starting.
I20260619 08:17:32.416723 18461 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34571
I20260619 08:17:32.416886 18610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34571 every 8 connection(s)
I20260619 08:17:32.417147 18461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:17:32.419116 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18461
I20260619 08:17:32.419413 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:35491
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=43989
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:17:32.420719 18503 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:32.423429 18503 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:32.423753 18503 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap complete.
I20260619 08:17:32.425004 18503 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260619 08:17:32.426410 18503 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.426712 18503 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.426855 18503 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.427104 18503 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:32.436668 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.417943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:50987 (local address 127.2.119.254:38433)
0619 08:17:32.421170 (+ 3227us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.421181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.421233 (+ 52us) server_negotiation.cc:407] Connection header received
0619 08:17:32.423787 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.423795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.423863 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.425046 (+ 1183us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.425136 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.426727 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.428398 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.429102 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.429230 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.432197 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.432201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.432201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.432215 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.434489 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.435028 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.435032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.435032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.435158 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.435455 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.435457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.435457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.435764 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.435839 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.436304 (+ 465us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.436469 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3169,"thread_start_us":91,"threads_started":1}
I20260619 08:17:32.436671 18613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.417859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:50987)
0619 08:17:32.418039 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.418088 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.418552 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.424154 (+ 5602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.424159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.424573 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.424893 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.424910 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.426850 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.426851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.428272 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.428276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.429218 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.429219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.429393 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.430075 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.430086 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.432050 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.434599 (+ 2549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.434607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.434609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.434927 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.435243 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.435244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.435245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.435376 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.435870 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.435880 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.436107 (+ 227us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.436231 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.436420 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260619 08:17:32.437678 18611 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:32.437788 18611 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:32.437983 18611 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:32.438782 18423 ts_manager.cc:194] Registered new tserver with Master: f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571)
I20260619 08:17:32.440008 18423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:50987
I20260619 08:17:32.440917 18611 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:17:32.448885 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.439731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.2.119.193:34571)
0619 08:17:32.440204 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.440208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.440217 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:17:32.440325 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.440328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.440344 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.440397 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.440942 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.442143 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.443234 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.443687 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.443758 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.446165 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.446172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.446172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.446178 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.447553 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.447989 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.447991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.447991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.448096 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.448292 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.448293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.448293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.448481 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.448547 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.448743 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.448820 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":77,"threads_started":1}
I20260619 08:17:32.448935 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.439452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.0.0.1:48532)
0619 08:17:32.439895 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.439942 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.440007 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.440462 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.440466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.440786 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.440845 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.440847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.442241 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.442242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.443149 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.443153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.443781 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.443782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.443856 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.444388 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.444394 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.446089 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.447633 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.447639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.447642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.447896 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.448162 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.448162 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.448163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.448231 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.448548 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.448555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.448607 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.448714 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.448830 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":95,"threads_started":1}
I20260619 08:17:32.449473 18545 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:48532
I20260619 08:17:32.450110 18620 tablet_replica.cc:330] stopping tablet replica
I20260619 08:17:32.450227 18620 raft_consensus.cc:2229] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:32.450302 18620 raft_consensus.cc:2258] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.450606 18620 ts_tablet_manager.cc:1905] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:32.452831 18620 ts_tablet_manager.cc:1918] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:17:32.452920 18620 log.cc:1196] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal/wals/4a4e8348427a485abf57a5bcaef33124
I20260619 08:17:32.453138 18620 ts_tablet_manager.cc:1939] T 4a4e8348427a485abf57a5bcaef33124 P f3f8b95bef3c4b10881002c4e3d489a2: Deleting consensus metadata
I20260619 08:17:32.453681 18410 catalog_manager.cc:4841] TS f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571): tablet 4a4e8348427a485abf57a5bcaef33124 (table test-table [id=d32c5d527a63404bbd045225665c4b20]) successfully deleted
W20260619 08:17:32.511160 18615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:32.511376 18615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:32.511407 18615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:32.512599 18615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:32.512642 18615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:32.512668 18615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:17:32.514163 18615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:35491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=43989
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:32.514430 18615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:32.514678 18615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:32.520434 18628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:32.520455 18615 server_base.cc:1034] running on GCE node
W20260619 08:17:32.520413 18625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.520408 18626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:32.521004 18615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:32.521260 18615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:32.522454 18615 hybrid_clock.cc:648] HybridClock initialized: now 1781857052522440 us; error 31 us; skew 500 ppm
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:32.525138 18615 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:17:32.525665 18615 webserver.cc:460] Webserver started at http://127.2.119.194:43989/ using document root <none> and password file <none>
I20260619 08:17:32.525866 18615 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:32.525915 18615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:32.527319 18615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:32.528127 18635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:32.528329 18615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:32.528414 18615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "1a904c89de62a538739d4bbcd0e5e337"
server_key_iv: "67b2413ec5c409309057087ecddd282d"
server_key_version: "encryptionkey@0"
I20260619 08:17:32.528784 18615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:32.546983 18615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:32.549892 18615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:32.550056 18615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:32.550256 18615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:32.550702 18642 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:17:32.552152 18615 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:17:32.552205 18615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:17:32.552232 18615 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:17:32.552886 18615 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:17:32.552917 18615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:32.553061 18642 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap starting.
I20260619 08:17:32.554948 18642 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:32.558364 18642 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:32.558693 18642 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap complete.
I20260619 08:17:32.558856 18642 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260619 08:17:32.559175 18615 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:35491
I20260619 08:17:32.559222 18750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:35491 every 8 connection(s)
I20260619 08:17:32.559520 18615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:17:32.560675 18642 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.561004 18642 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c39b3934c1f4d3dbf178dff3039ce0a, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.561152 18642 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.561444 18642 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:32.567440 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18615
I20260619 08:17:32.567723 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:33755
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=34823
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:17:32.569720 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.560284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:58465 (local address 127.2.119.254:38433)
0619 08:17:32.560346 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.560349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.560585 (+ 236us) server_negotiation.cc:407] Connection header received
0619 08:17:32.560763 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.560765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.560779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.560832 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.561718 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.562244 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.563227 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.563445 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.563525 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.566342 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.566343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.566344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.566351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.567902 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.568509 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.568510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.568510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.568571 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.568893 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.568894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.568894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.569035 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.569145 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.569510 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.569617 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:17:32.569957 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.560192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:58465)
0619 08:17:32.560421 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.560471 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.560651 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.560900 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.560904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.561358 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.561627 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.561632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.562348 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.562349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.563127 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.563129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.563557 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.563558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.563669 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.564268 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.564277 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.566244 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.568043 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.568052 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.568056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.568404 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.568649 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.568649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.568650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.568805 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.569136 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.569143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.569350 (+ 207us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.569486 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.569691 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260619 08:17:32.570525 18751 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:32.570627 18751 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:32.570834 18751 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:32.571408 18423 ts_manager.cc:194] Registered new tserver with Master: 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
I20260619 08:17:32.572058 18423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:58465
I20260619 08:17:32.573909 18751 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.194@KRBTEST.COM
I20260619 08:17:32.580624 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.571743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.0.0.1:45192)
0619 08:17:32.572216 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.572229 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.572285 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.572793 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.572794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.572992 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.573060 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.573063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.574209 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.574209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.574904 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.574907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.575422 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.575423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.575496 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.576313 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.576318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.577606 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.579475 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.579476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.579477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.579657 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.579889 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.579890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.579890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.579958 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.580268 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.580269 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.580319 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.580401 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.580519 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454}
I20260619 08:17:32.580688 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.572381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45192 (local address 127.2.119.194:35491)
0619 08:17:32.572615 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.572620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.572632 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:17:32.572666 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.572668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.572683 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.572769 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.573178 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.574015 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.575000 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.575316 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.575436 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.577700 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.577709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.577710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.577717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.579367 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.579751 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.579755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.579755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.579816 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.580023 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.580024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.580024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.580188 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.580277 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.580452 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.580546 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260619 08:17:32.581202 18678 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:45192
I20260619 08:17:32.581882 18757 tablet_replica.cc:330] stopping tablet replica
I20260619 08:17:32.582005 18757 raft_consensus.cc:2229] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:32.582077 18757 raft_consensus.cc:2258] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.582409 18757 ts_tablet_manager.cc:1905] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:32.584935 18757 ts_tablet_manager.cc:1918] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:17:32.585016 18757 log.cc:1196] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/4a4e8348427a485abf57a5bcaef33124
I20260619 08:17:32.585268 18757 ts_tablet_manager.cc:1939] T 4a4e8348427a485abf57a5bcaef33124 P 5c39b3934c1f4d3dbf178dff3039ce0a: Deleting consensus metadata
I20260619 08:17:32.585896 18409 catalog_manager.cc:4841] TS 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491): tablet 4a4e8348427a485abf57a5bcaef33124 (table test-table [id=d32c5d527a63404bbd045225665c4b20]) successfully deleted
W20260619 08:17:32.656131 18755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:32.656328 18755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:32.656350 18755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:32.657522 18755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:32.657564 18755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:32.657591 18755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:17:32.659054 18755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:33755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=34823
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:32.659298 18755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:32.659533 18755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:32.664749 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.664824 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:32.664749 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:32.664868 18755 server_base.cc:1034] running on GCE node
I20260619 08:17:32.665355 18755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:32.665674 18755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:32.666850 18755 hybrid_clock.cc:648] HybridClock initialized: now 1781857052666825 us; error 41 us; skew 500 ppm
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:32.669490 18755 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:17:32.670017 18755 webserver.cc:460] Webserver started at http://127.2.119.195:34823/ using document root <none> and password file <none>
I20260619 08:17:32.670280 18755 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:32.670337 18755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:32.671715 18755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:32.672508 18772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:32.672703 18755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:32.672804 18755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "915d4aa7921c48d5911715fc6c8c6686"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "2f63f6028645888a032720f8a240a070"
server_key_iv: "209d908a63075d4ec85bdd573b726865"
server_key_version: "encryptionkey@0"
I20260619 08:17:32.673178 18755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:32.685958 18755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:32.688561 18755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:32.688732 18755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:32.688982 18755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:32.689515 18779 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:17:32.691044 18755 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:17:32.691128 18755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:17:32.691177 18755 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:17:32.691839 18755 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:17:32.691897 18755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:32.692026 18779 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap starting.
I20260619 08:17:32.693758 18779 log.cc:824] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:32.697280 18779 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:32.697649 18779 tablet_bootstrap.cc:492] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap complete.
I20260619 08:17:32.697830 18779 ts_tablet_manager.cc:1397] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260619 08:17:32.699079 18755 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33755
I20260619 08:17:32.699155 18887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33755 every 8 connection(s)
I20260619 08:17:32.699465 18755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:17:32.699985 18779 raft_consensus.cc:348] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.700313 18779 raft_consensus.cc:729] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915d4aa7921c48d5911715fc6c8c6686, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.700466 18779 consensus_queue.cc:260] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } } peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } }
I20260619 08:17:32.700743 18779 ts_tablet_manager.cc:1428] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:17:32.704017 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18755
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:17:32.709764 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.700200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:52861 (local address 127.2.119.254:38433)
0619 08:17:32.700275 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.700277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.700588 (+ 311us) server_negotiation.cc:407] Connection header received
0619 08:17:32.700749 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.700750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.700762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.700832 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.701631 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.702148 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.703177 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.703402 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.703495 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.706523 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.706524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.706524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.706533 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.708098 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.708538 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.708539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.708547 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.708620 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.708916 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.708916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.708917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.709077 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.709158 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.709546 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.709648 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:17:32.709847 18890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.700183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:52861)
0619 08:17:32.700426 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.700477 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.700651 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.700910 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.700914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.701247 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.701534 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.701539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.702262 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.702263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.703078 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.703080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.703504 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.703505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.703642 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.704326 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.704334 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.706426 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.708203 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.708210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.708213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.708454 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.708694 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.708694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.708695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.708844 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.709154 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.709159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.709290 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.709465 (+ 175us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.709651 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
I20260619 08:17:32.710424 18888 heartbeater.cc:347] Connected to a master server at 127.2.119.254:38433
I20260619 08:17:32.710551 18888 heartbeater.cc:464] Registering TS with master...
I20260619 08:17:32.710794 18888 heartbeater.cc:511] Master 127.2.119.254:38433 requested a full tablet report, sending...
I20260619 08:17:32.711473 18423 ts_manager.cc:194] Registered new tserver with Master: 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755)
I20260619 08:17:32.711701 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.704513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.2.119.254:38433)
0619 08:17:32.704643 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.704645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.704756 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:17:32.704789 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.704790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.704803 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.704879 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.705503 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.706042 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.706989 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.707207 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.707293 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.708562 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.708564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.708564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.708571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.710292 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.710746 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.710746 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.710747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.710797 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.711092 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.711093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.711093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.711242 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.711397 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.711500 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.711621 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260619 08:17:32.712076 18423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:52861
I20260619 08:17:32.712751 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:17:32.713184 18888 heartbeater.cc:503] Master 127.2.119.254:38433 was elected leader, sending a full tablet report...
I20260619 08:17:32.715813 18893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.713151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.2.119.254:38433)
0619 08:17:32.713196 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.713199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.713366 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:17:32.713441 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.713442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.713474 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.713532 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:17:32.713752 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.714242 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.715005 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.715223 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.715307 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.715488 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:17:32.715594 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:17:32.715688 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.715707 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.715730 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Jun 19 08:17:32 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.195@KRBTEST.COM
I20260619 08:17:32.716089 18423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:17:32.718020 18423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:17:32.720814 18545 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 (DEFAULT_TABLE table=test-table [id=8f05e0b086f8491b9789c19e16d3906d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:32.720818 18678 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 (DEFAULT_TABLE table=test-table [id=8f05e0b086f8491b9789c19e16d3906d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:32.720862 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.711783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.0.0.1:58118)
0619 08:17:32.711823 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.711836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.711878 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.712577 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.712578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.712749 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:32.712791 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.712793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.714156 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.714157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.714711 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.714712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.715285 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.715287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.715350 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.715828 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:32.715834 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:32.717214 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:32.719607 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.719608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.719609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.719885 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.720132 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:32.720133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:32.720134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:32.720208 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:32.720544 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:32.720545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:32.720594 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.720665 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.720771 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 08:17:32.720999 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9c10a4de94fdaa95087690194c5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:32.721014 18545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9c10a4de94fdaa95087690194c5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:32.722362 18642 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap starting.
I20260619 08:17:32.722561 18503 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap starting.
I20260619 08:17:32.722805 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.712256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.2.119.195:33755)
0619 08:17:32.712413 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.712418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.712427 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:17:32.712458 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.712460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.712474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.712549 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:32.712905 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.714070 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.714820 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.715187 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.715273 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.717342 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:32.717350 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:32.717351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:32.717358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:32.719507 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.719972 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.719975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.719975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.720031 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:32.720280 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:32.720281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:32.720281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:32.720482 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:32.720566 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.722556 (+ 1990us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.722696 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260619 08:17:32.722838 18642 tablet_bootstrap.cc:654] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:32.722867 18503 tablet_bootstrap.cc:654] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:32.723187 18822 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a4e8348427a485abf57a5bcaef33124 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:12 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58118
I20260619 08:17:32.723740 18642 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: No bootstrap required, opened a new log
I20260619 08:17:32.723790 18642 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:32.723925 18898 tablet_replica.cc:330] stopping tablet replica
I20260619 08:17:32.723958 18642 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.723994 18821 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 (DEFAULT_TABLE table=test-table [id=8f05e0b086f8491b9789c19e16d3906d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:17:32.724035 18642 raft_consensus.cc:374] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:32.724054 18898 raft_consensus.cc:2229] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:32.724084 18642 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c39b3934c1f4d3dbf178dff3039ce0a, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.724124 18898 raft_consensus.cc:2258] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.724143 18642 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.724174 18821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9c10a4de94fdaa95087690194c5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:32.724254 18642 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:17:32.724618 18898 ts_tablet_manager.cc:1905] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:32.725171 18503 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: No bootstrap required, opened a new log
I20260619 08:17:32.725239 18503 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 08:17:32.725412 18503 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.725504 18503 raft_consensus.cc:374] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:32.725557 18503 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.725610 18503 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.725718 18503 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 08:17:32.726642 18779 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap starting.
I20260619 08:17:32.726931 18898 ts_tablet_manager.cc:1918] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:17:32.727008 18898 log.cc:1196] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/4a4e8348427a485abf57a5bcaef33124
I20260619 08:17:32.727064 18779 tablet_bootstrap.cc:654] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Neither blocks nor log segments found. Creating new log.
I20260619 08:17:32.727330 18898 ts_tablet_manager.cc:1939] T 4a4e8348427a485abf57a5bcaef33124 P 915d4aa7921c48d5911715fc6c8c6686: Deleting consensus metadata
I20260619 08:17:32.727782 18407 catalog_manager.cc:4841] TS 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755): tablet 4a4e8348427a485abf57a5bcaef33124 (table test-table [id=d32c5d527a63404bbd045225665c4b20]) successfully deleted
I20260619 08:17:32.727945 18779 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: No bootstrap required, opened a new log
I20260619 08:17:32.728010 18779 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:32.728164 18779 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.728246 18779 raft_consensus.cc:374] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:17:32.728302 18779 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915d4aa7921c48d5911715fc6c8c6686, State: Initialized, Role: FOLLOWER
I20260619 08:17:32.728417 18779 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.728554 18779 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:32.752174 18617 raft_consensus.cc:482] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:17:32.752308 18617 raft_consensus.cc:504] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.752625 18617 leader_election.cc:288] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755), 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
I20260619 08:17:32.756059 18894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.752855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46727 (local address 127.2.119.195:33755)
0619 08:17:32.752900 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.752902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.752913 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:17:32.752952 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.752952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.752976 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.753035 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:32.753260 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.753844 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.755086 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.755637 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.755692 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.755750 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.755949 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.755983 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:17:32.756067 18613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.752771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.2.119.193:46727)
0619 08:17:32.752813 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.752827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.752881 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.753063 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.753065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.753067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:32.753158 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.753160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.753932 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.753933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.754998 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.755000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.755731 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.755732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.755832 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.755858 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.755935 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.755974 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 08:17:32.756399 18842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915d4aa7921c48d5911715fc6c8c6686" is_pre_election: true
I20260619 08:17:32.756584 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.752819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.2.119.193:44531)
0619 08:17:32.753051 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.753062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:32.753120 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:32.753266 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:32.753266 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:32.753268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:17:32.753364 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.753365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.754529 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.754530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.755402 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:32.755405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.756307 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.756308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:32.756385 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.756389 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:17:32.756447 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:17:32.756485 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
I20260619 08:17:32.756582 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.752911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:44531 (local address 127.2.119.194:35491)
0619 08:17:32.752959 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.752961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.753126 (+ 165us) server_negotiation.cc:407] Connection header received
0619 08:17:32.753158 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.753159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.753184 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.753252 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:17:32.753437 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.754427 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.755504 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.756167 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.756241 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.756307 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.756469 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.756513 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:17:32.756593 18842 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 0.
I20260619 08:17:32.756817 18497 leader_election.cc:302] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915d4aa7921c48d5911715fc6c8c6686, f3f8b95bef3c4b10881002c4e3d489a2; no voters:
I20260619 08:17:32.756919 18704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" is_pre_election: true
I20260619 08:17:32.757009 18617 raft_consensus.cc:2781] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:17:32.757077 18617 raft_consensus.cc:482] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:17:32.757078 18704 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 0.
I20260619 08:17:32.757107 18617 raft_consensus.cc:3037] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:32.757874 18617 raft_consensus.cc:504] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.758030 18617 leader_election.cc:288] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 1 election: Requested vote from peers 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755), 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
I20260619 08:17:32.758188 18704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
I20260619 08:17:32.758270 18842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915d4aa7921c48d5911715fc6c8c6686"
I20260619 08:17:32.758275 18704 raft_consensus.cc:3037] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:32.758350 18842 raft_consensus.cc:3037] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:17:32.758975 18842 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 1.
I20260619 08:17:32.758978 18704 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 1.
I20260619 08:17:32.759179 18497 leader_election.cc:302] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915d4aa7921c48d5911715fc6c8c6686, f3f8b95bef3c4b10881002c4e3d489a2; no voters:
I20260619 08:17:32.759264 18617 raft_consensus.cc:2781] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:17:32.759444 18617 raft_consensus.cc:686] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 LEADER]: Becoming Leader. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Running, Role: LEADER
I20260619 08:17:32.759528 18617 consensus_queue.cc:237] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:32.760289 18423 catalog_manager.cc:5495] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } health_report { overall_health: UNKNOWN } } }
I20260619 08:17:32.772442 18619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.770080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.2.119.193:34571)
0619 08:17:32.770138 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:32.770140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:32.770154 (+ 14us) server_negotiation.cc:407] Connection header received
0619 08:17:32.770199 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:32.770200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:32.770221 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:32.770336 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:17:32.770476 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.770958 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.771666 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:32.771872 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:32.771929 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:32.772155 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:17:32.772254 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:17:32.772328 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:32.772364 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:17:32.772384 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 08:17:32.774093 18842 raft_consensus.cc:1261] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Refusing update from remote peer f3f8b95bef3c4b10881002c4e3d489a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:17:32.774121 18704 raft_consensus.cc:1261] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Refusing update from remote peer f3f8b95bef3c4b10881002c4e3d489a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:17:32.774458 18901 consensus_queue.cc:1035] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:17:32.774595 18617 consensus_queue.cc:1035] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:17:32.776690 18904 mvcc.cc:205] Tried to move back new op lower bound from 7298486488160120832 to 7298486488103682048. Current Snapshot: MvccSnapshot[applied={T|T < 7298486488160120832}]
I20260619 08:17:32.777556 18905 mvcc.cc:205] Tried to move back new op lower bound from 7298486488160120832 to 7298486488103682048. Current Snapshot: MvccSnapshot[applied={T|T < 7298486488160120832}]
I20260619 08:17:32.777727 18906 mvcc.cc:205] Tried to move back new op lower bound from 7298486488160120832 to 7298486488103682048. Current Snapshot: MvccSnapshot[applied={T|T < 7298486488160120832}]
I20260619 08:17:32.778837 18422 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:17:32.778941 18422 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:17:32.779692 18422 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Sending DeleteTablet for 3 replicas of tablet 5fa9c10a4de94fdaa95087690194c5b1
I20260619 08:17:32.779976 18821 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:32 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58118
I20260619 08:17:32.780072 18898 tablet_replica.cc:330] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: stopping tablet replica
I20260619 08:17:32.780069 18678 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:32 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:45192
I20260619 08:17:32.780123 18898 raft_consensus.cc:2229] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:32.780164 18898 raft_consensus.cc:2258] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.780249 18757 tablet_replica.cc:330] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: stopping tablet replica
I20260619 08:17:32.780313 18757 raft_consensus.cc:2229] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:17:32.780366 18757 raft_consensus.cc:2258] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.780443 18545 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa9c10a4de94fdaa95087690194c5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:17:32 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:48532
I20260619 08:17:32.780531 18620 tablet_replica.cc:330] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: stopping tablet replica
I20260619 08:17:32.780580 18620 raft_consensus.cc:2229] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:17:32.780584 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18461
I20260619 08:17:32.780658 18620 raft_consensus.cc:2258] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:17:32.782075 18898 ts_tablet_manager.cc:1905] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:32.783644 18898 ts_tablet_manager.cc:1918] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:17:32.783703 18898 log.cc:1196] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/5fa9c10a4de94fdaa95087690194c5b1
I20260619 08:17:32.783918 18898 ts_tablet_manager.cc:1939] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Deleting consensus metadata
I20260619 08:17:32.784228 18407 catalog_manager.cc:4841] TS 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755): tablet 5fa9c10a4de94fdaa95087690194c5b1 (table test-table [id=8f05e0b086f8491b9789c19e16d3906d]) successfully deleted
W20260619 08:17:32.786710 18410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:34571 (error 108)
I20260619 08:17:32.787019 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18615
I20260619 08:17:32.787139 18757 ts_tablet_manager.cc:1905] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:17:32.787765 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.787355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.0.0.1:48542)
0619 08:17:32.787678 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.787727 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
W20260619 08:17:32.788116 18410 catalog_manager.cc:4584] TS f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5fa9c10a4de94fdaa95087690194c5b1: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34571: connect: Connection refused (error 111)
I20260619 08:17:32.793270 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18755
I20260619 08:17:32.793473 18618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:32.793320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.0.0.1:45208)
0619 08:17:32.793392 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:32.793425 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:35491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 08:17:32.802280 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18390
I20260619 08:17:52.808063 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--webserver_interface=127.2.119.254
--webserver_port=33043
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:38433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:52.888808 18914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:52.889017 18914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:52.889039 18914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:52.890148 18914 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 08:17:52.890183 18914 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 08:17:52.890196 18914 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 08:17:52.890208 18914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:17:52.890221 18914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:52.890235 18914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:52.890247 18914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:17:52.890259 18914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:17:52.891634 18914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:38433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:38433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=33043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:52.891813 18914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:52.891980 18914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:52.896972 18922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:52.896958 18920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:52.896944 18919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:52.897097 18914 server_base.cc:1034] running on GCE node
I20260619 08:17:52.897444 18914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:52.897687 18914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:52.898849 18914 hybrid_clock.cc:648] HybridClock initialized: now 1781857072898833 us; error 29 us; skew 500 ppm
Jun 19 08:17:52 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857072, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:52.901271 18914 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:17:52.901741 18914 webserver.cc:460] Webserver started at http://127.2.119.254:33043/ using document root <none> and password file <none>
I20260619 08:17:52.901973 18914 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:52.902043 18914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:52.903332 18914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:52.904008 18929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:52.904269 18914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:52.904413 18914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "766543abf11b4ed1865250709dbaeb6f"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "446b72067aa86f2fc3fe526e88c07f7b"
server_key_iv: "f0ec8c812d2184ecf55aeaa23377c82a"
server_key_version: "encryptionkey@0"
I20260619 08:17:52.904765 18914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:52.926537 18914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:52.928422 18914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:52.928591 18914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:52.932649 18914 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:38433
I20260619 08:17:52.932693 18981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:38433 every 8 connection(s)
I20260619 08:17:52.933099 18914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:17:52.934618 18982 sys_catalog.cc:263] Verifying existing consensus state
I20260619 08:17:52.935272 18982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap starting.
I20260619 08:17:52.943349 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18914
I20260619 08:17:52.943683 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:34571
--local_ip_for_outbound_sockets=127.2.119.193
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=38157
--webserver_interface=127.2.119.193
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:17:52.943956 18982 log.cc:824] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:52.946417 18982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:52.946753 18982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Bootstrap complete.
I20260619 08:17:52.949013 18982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:52.949234 18982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Initialized, Role: FOLLOWER
I20260619 08:17:52.949432 18982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:52.949532 18982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:17:52.949589 18982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:17:52.949643 18982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 2 FOLLOWER]: Advancing to term 3
I20260619 08:17:52.950644 18982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:52.950767 18982 leader_election.cc:302] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 766543abf11b4ed1865250709dbaeb6f; no voters:
I20260619 08:17:52.950970 18982 leader_election.cc:288] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 08:17:52.951012 18986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 08:17:52.951129 18986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [term 3 LEADER]: Becoming Leader. State: Replica: 766543abf11b4ed1865250709dbaeb6f, State: Running, Role: LEADER
I20260619 08:17:52.951215 18986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } }
I20260619 08:17:52.951241 18982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:17:52.951555 18986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 766543abf11b4ed1865250709dbaeb6f. Latest consensus state: current_term: 3 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:52.951639 18986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:52.951560 18987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "766543abf11b4ed1865250709dbaeb6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "766543abf11b4ed1865250709dbaeb6f" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 38433 } } }
I20260619 08:17:52.951843 18987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f [sys.catalog]: This master's current role is: LEADER
I20260619 08:17:52.951927 18991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:17:52.952776 18991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:17:52.952912 18991 catalog_manager.cc:1259] Loaded cluster ID: 27ddbff3008546f7b53fa6a510b246cb
I20260619 08:17:52.953140 18991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:17:52.954030 18991 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:17:52.954224 18991 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Loaded TSK: 0
I20260619 08:17:52.954282 18991 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 766543abf11b4ed1865250709dbaeb6f: Loaded TSK: 1
F20260619 08:17:52.954487 18991 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc7b9c7c9ad google::LogMessage::Fail() at ??:0
@ 0x7fc7b9c7f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc7b9c7c4de google::LogMessage::Flush() at ??:0
@ 0x7fc7b9c7da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc7bc3e9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc7bc3e6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc7c1bd61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc7c1bd648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc7c1bd6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc7c1b9cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc7c1ba6ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc7bbb4a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc7bbb3f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc7bc9a36db start_thread at ??:0
@ 0x7fc7b877a61f clone at ??:0
W20260619 08:17:53.066190 18985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:53.066457 18985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:53.066493 18985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:53.068241 18985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:53.068298 18985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:53.068341 18985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:17:53.070497 18985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:34571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=38157
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:53.070750 18985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:53.070990 18985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:53.076339 19026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:53.076463 19024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:53.076372 19023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:53.076589 18985 server_base.cc:1034] running on GCE node
I20260619 08:17:53.076860 18985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:53.077152 18985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:53.078346 18985 hybrid_clock.cc:648] HybridClock initialized: now 1781857073078339 us; error 42 us; skew 500 ppm
Jun 19 08:17:53 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:53.081503 18985 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:17:53.081920 18985 webserver.cc:460] Webserver started at http://127.2.119.193:38157/ using document root <none> and password file <none>
I20260619 08:17:53.082232 18985 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:53.082304 18985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:53.083742 18985 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:17:53.084542 19033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:53.084749 18985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:53.084848 18985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "f3f8b95bef3c4b10881002c4e3d489a2"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "54bb4fb8edfc72f27d8ea6eb7a55f378"
server_key_iv: "9becf04fac572391b7ffbdc93b09fe2d"
server_key_version: "encryptionkey@0"
I20260619 08:17:53.085244 18985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:53.101416 18985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:53.103726 18985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:53.103892 18985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:53.104131 18985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:53.104583 19040 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:17:53.106629 18985 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:17:53.106698 18985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:17:53.106738 18985 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:17:53.107386 18985 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:17:53.107443 18985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:53.107524 19040 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap starting.
I20260619 08:17:53.114647 18985 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34571
I20260619 08:17:53.114799 19147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34571 every 8 connection(s)
I20260619 08:17:53.115062 18985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:17:53.118387 19040 log.cc:824] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:53.119829 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 18985
I20260619 08:17:53.120158 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:35491
--local_ip_for_outbound_sockets=127.2.119.194
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=43989
--webserver_interface=127.2.119.194
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:17:53.121047 19040 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:53.121377 19040 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Bootstrap complete.
I20260619 08:17:53.122627 19040 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.010s
I20260619 08:17:53.124150 19040 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:53.124487 19040 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Initialized, Role: FOLLOWER
I20260619 08:17:53.124660 19040 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:53.124931 19040 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260619 08:17:53.177407 19150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:17:53.115734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:42707)
0619 08:17:53.115922 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.115963 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:53.116408 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:53.177163 (+ 60755us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
W20260619 08:17:53.177793 19148 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:38433 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:38433: Client connection negotiation failed: client connection to 127.2.119.254:38433: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 08:17:53.178184 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.178057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:58993)
0619 08:17:53.178114 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.178161 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:17:53.178601 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.178505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:57195)
0619 08:17:53.178549 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.178573 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:17:53.178735 19148 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 08:17:53.204309 19152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:53.204499 19152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:53.204522 19152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:53.205749 19152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:53.205790 19152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:53.205817 19152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:17:53.207290 19152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:35491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=43989
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:53.207536 19152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:53.207769 19152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:53.212838 19158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:53.212863 19161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:53.212831 19159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:53.213068 19152 server_base.cc:1034] running on GCE node
I20260619 08:17:53.213258 19152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:53.213526 19152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:53.214674 19152 hybrid_clock.cc:648] HybridClock initialized: now 1781857073214658 us; error 33 us; skew 500 ppm
Jun 19 08:17:53 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:53.217105 19152 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:17:53.217604 19152 webserver.cc:460] Webserver started at http://127.2.119.194:43989/ using document root <none> and password file <none>
I20260619 08:17:53.217849 19152 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:53.217893 19152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:53.219192 19152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:53.220070 19168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:53.220253 19152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:53.220355 19152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "1a904c89de62a538739d4bbcd0e5e337"
server_key_iv: "67b2413ec5c409309057087ecddd282d"
server_key_version: "encryptionkey@0"
I20260619 08:17:53.220762 19152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:53.230677 19152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:53.232604 19152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:53.232767 19152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:53.232996 19152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:53.233502 19175 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 08:17:53.234999 19152 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 08:17:53.235052 19152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 08:17:53.235106 19152 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 08:17:53.235739 19152 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 08:17:53.235810 19152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 08:17:53.235965 19175 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap starting.
I20260619 08:17:53.237614 19175 log.cc:824] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:53.240072 19175 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:53.240355 19175 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Bootstrap complete.
I20260619 08:17:53.240528 19175 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260619 08:17:53.242033 19175 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:53.242306 19175 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c39b3934c1f4d3dbf178dff3039ce0a, State: Initialized, Role: FOLLOWER
I20260619 08:17:53.242498 19175 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:53.242830 19175 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
I20260619 08:17:53.243489 19152 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:35491
I20260619 08:17:53.243533 19284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:35491 every 8 connection(s)
I20260619 08:17:53.243841 19152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:17:53.244832 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.244464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:55213)
0619 08:17:53.244632 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.244717 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
W20260619 08:17:53.245251 19285 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:38433 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:38433: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
I20260619 08:17:53.245589 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.245488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:38961)
0619 08:17:53.245539 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.245570 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:17:53.245823 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19152
I20260619 08:17:53.245932 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.245844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:57871)
0619 08:17:53.245879 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.245912 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 08:17:53.246030 19285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:17:53.246117 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:33755
--local_ip_for_outbound_sockets=127.2.119.195
--tserver_master_addrs=127.2.119.254:38433
--webserver_port=34823
--webserver_interface=127.2.119.195
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:17:53.329084 19288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:17:53.329281 19288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:17:53.329357 19288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:17:53.330665 19288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:17:53.330705 19288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:17:53.330732 19288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:17:53.332158 19288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:33755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=34823
--tserver_master_addrs=127.2.119.254:38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:17:53.332396 19288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:17:53.332620 19288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:17:53.337653 19296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:53.337668 19288 server_base.cc:1034] running on GCE node
W20260619 08:17:53.337653 19294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:17:53.337690 19293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:17:53.338052 19288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:17:53.338294 19288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:17:53.339457 19288 hybrid_clock.cc:648] HybridClock initialized: now 1781857073339436 us; error 36 us; skew 500 ppm
Jun 19 08:17:53 dist-test-slave-d0b5 krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:17:53.342113 19288 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:17:53.342597 19288 webserver.cc:460] Webserver started at http://127.2.119.195:34823/ using document root <none> and password file <none>
I20260619 08:17:53.342841 19288 fs_manager.cc:359] Metadata directory not provided
I20260619 08:17:53.342919 19288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:17:53.344421 19288 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:17:53.345031 19303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:53.345209 19288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:17:53.345329 19288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "915d4aa7921c48d5911715fc6c8c6686"
format_stamp: "Formatted at 2026-06-19 08:17:11 on dist-test-slave-d0b5"
server_key: "2f63f6028645888a032720f8a240a070"
server_key_iv: "209d908a63075d4ec85bdd573b726865"
server_key_version: "encryptionkey@0"
I20260619 08:17:53.345685 19288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:17:53.368702 19288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:17:53.370800 19288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:17:53.370980 19288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:17:53.371227 19288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:17:53.371541 19288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:17:53.371596 19288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:53.371635 19288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:17:53.371665 19288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:17:53.377665 19288 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:33755
I20260619 08:17:53.377714 19416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:33755 every 8 connection(s)
I20260619 08:17:53.378052 19288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:17:53.379071 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.378668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:40833)
0619 08:17:53.378814 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.378923 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
W20260619 08:17:53.379484 19417 heartbeater.cc:644] Failed to heartbeat to 127.2.119.254:38433 (0 consecutive failures): Network error: Failed to ping master at 127.2.119.254:38433: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
I20260619 08:17:53.379808 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.379718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:37135)
0619 08:17:53.379765 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.379790 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:17:53.380079 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:53.380006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:42299)
0619 08:17:53.380044 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:53.380064 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:17:53.380200 19417 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 08:17:53.381186 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19288
I20260619 08:17:54.179379 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.179131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:56159)
0619 08:17:54.179265 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.179316 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260619 08:17:54.246627 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.246400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:39639)
0619 08:17:54.246539 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.246579 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260619 08:17:54.380774 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.380554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:54531)
0619 08:17:54.380686 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.380724 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260619 08:17:54.532198 19424 raft_consensus.cc:482] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:17:54.532323 19424 raft_consensus.cc:504] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:54.532781 19424 leader_election.cc:288] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755), 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
Jun 19 08:17:54 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.195@KRBTEST.COM
Jun 19 08:17:54 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.194@KRBTEST.COM
I20260619 08:17:54.551512 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.532946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:33755 (local address 127.2.119.193:36379)
0619 08:17:54.532998 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.533009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:54.533065 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:54.534051 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:54.534052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:54.534223 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:54.534459 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:54.534466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.536105 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.536105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:54.537429 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:54.537434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.538281 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.538282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:54.538466 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:54.544976 (+ 6510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:54.544991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:54.547043 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:54.549552 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:54.549560 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:54.549562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:54.549861 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:54.550104 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:54.550105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:54.550254 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:54.550932 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:54.550939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:54.551105 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 08:17:54.551215 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 08:17:54.551394 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":89}
I20260619 08:17:54.551570 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.533082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:36379 (local address 127.2.119.195:33755)
0619 08:17:54.533194 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:54.533206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:54.533244 (+ 38us) server_negotiation.cc:407] Connection header received
0619 08:17:54.533898 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:54.533901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:54.533939 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:54.534038 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:54.534590 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.535973 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.537544 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.538166 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.538281 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:54.547210 (+ 8929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:54.547215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:54.547216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:54.547228 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:54.549466 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:54.549945 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:54.549948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:54.549949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:54.550034 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550568 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:54.550569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:54.550569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:54.550845 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:54.550960 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:54.551285 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 08:17:54.551457 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":54,"threads_started":1}
I20260619 08:17:54.551766 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.532959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35491 (local address 127.2.119.193:40423)
0619 08:17:54.533100 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.533110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:54.533156 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:54.533798 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:54.533800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:54.534182 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:54.534459 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:54.534466 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.536103 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.536104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:54.537436 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:54.537437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.538282 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.538283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:54.538466 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:54.544976 (+ 6510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:54.544991 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:54.547360 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:54.550329 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:54.550330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:54.550498 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:54.550738 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:54.550739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:54.550800 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:54.551233 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:54.551234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:54.551289 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 08:17:54.551365 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 08:17:54.551505 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":52,"thread_start_us":57,"threads_started":1}
I20260619 08:17:54.552242 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.533197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:40423 (local address 127.2.119.194:35491)
0619 08:17:54.533410 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:54.533417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:54.533437 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:17:54.533524 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:54.533526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:54.533578 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:54.533716 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:54.534584 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.535972 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.537544 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:54.538166 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:54.538283 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:54.547488 (+ 9205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:54.547494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:54.547494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:54.547518 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:54.550245 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550569 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:54.550571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:54.550571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:54.550653 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:54.550867 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:54.550868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:54.550869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:54.551147 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:54.551238 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:54.551436 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 08:17:54.552142 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
W20260619 08:17:54.552479 19034 leader_election.cc:341] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755): Not found: Tablet not found: 5fa9c10a4de94fdaa95087690194c5b1
I20260619 08:17:54.552692 19238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" is_pre_election: true
I20260619 08:17:54.552863 19238 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 1.
I20260619 08:17:54.553148 19036 leader_election.cc:302] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c39b3934c1f4d3dbf178dff3039ce0a, f3f8b95bef3c4b10881002c4e3d489a2; no voters: 915d4aa7921c48d5911715fc6c8c6686
I20260619 08:17:54.553361 19424 raft_consensus.cc:2781] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 08:17:54.553455 19424 raft_consensus.cc:482] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:17:54.553474 19424 raft_consensus.cc:3037] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:17:54.554800 19424 raft_consensus.cc:504] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:54.554996 19424 leader_election.cc:288] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 election: Requested vote from peers 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755), 5c39b3934c1f4d3dbf178dff3039ce0a (127.2.119.194:35491)
W20260619 08:17:54.555197 19034 leader_election.cc:341] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755): Not found: Tablet not found: 5fa9c10a4de94fdaa95087690194c5b1
I20260619 08:17:54.555192 19238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa9c10a4de94fdaa95087690194c5b1" candidate_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a"
I20260619 08:17:54.555279 19238 raft_consensus.cc:3037] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 1 FOLLOWER]: Advancing to term 2
I20260619 08:17:54.556507 19238 raft_consensus.cc:2454] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3f8b95bef3c4b10881002c4e3d489a2 in term 2.
I20260619 08:17:54.556699 19036 leader_election.cc:302] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c39b3934c1f4d3dbf178dff3039ce0a, f3f8b95bef3c4b10881002c4e3d489a2; no voters: 915d4aa7921c48d5911715fc6c8c6686
I20260619 08:17:54.556793 19424 raft_consensus.cc:2781] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 08:17:54.556936 19424 raft_consensus.cc:686] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [term 2 LEADER]: Becoming Leader. State: Replica: f3f8b95bef3c4b10881002c4e3d489a2, State: Running, Role: LEADER
I20260619 08:17:54.557037 19424 consensus_queue.cc:237] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:54.557511 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:54.557142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:38659)
0619 08:17:54.557460 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:54.557489 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306}
W20260619 08:17:55.007535 19034 consensus_peers.cc:487] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 -> Peer 915d4aa7921c48d5911715fc6c8c6686 (127.2.119.195:33755): Couldn't send request to peer 915d4aa7921c48d5911715fc6c8c6686. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fa9c10a4de94fdaa95087690194c5b1. This is attempt 1: this message will repeat every 5th retry.
I20260619 08:17:55.019375 19238 raft_consensus.cc:1261] T 5fa9c10a4de94fdaa95087690194c5b1 P 5c39b3934c1f4d3dbf178dff3039ce0a [term 2 FOLLOWER]: Refusing update from remote peer f3f8b95bef3c4b10881002c4e3d489a2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260619 08:17:55.019873 19424 consensus_queue.cc:1035] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 08:17:55.020923 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.020723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:55245)
0619 08:17:55.020829 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.020875 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260619 08:17:55.021670 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.021565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:53103)
0619 08:17:55.021609 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.021635 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:17:55.021946 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.021844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:51667)
0619 08:17:55.021886 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.021913 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 08:17:55.381599 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.381349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:51931)
0619 08:17:55.381470 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.381551 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":80,"threads_started":1}
I20260619 08:17:55.474704 19436 ts_tablet_manager.cc:927] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Initiating tablet copy from peer f3f8b95bef3c4b10881002c4e3d489a2 (127.2.119.193:34571)
I20260619 08:17:55.474927 19436 tablet_copy_client.cc:316] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: tablet copy: Beginning tablet copy session from remote peer at address 127.2.119.193:34571
Jun 19 08:17:55 dist-test-slave-d0b5 krb5kdc[17846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857073, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.193@KRBTEST.COM
I20260619 08:17:55.483383 19438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.475365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:48883 (local address 127.2.119.193:34571)
0619 08:17:55.475482 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 08:17:55.475486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:17:55.475495 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:17:55.475522 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:17:55.475524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:17:55.475536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:17:55.475595 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:17:55.476076 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:55.476793 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:55.477624 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:17:55.477948 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:17:55.478019 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:55.480035 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:17:55.480042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:17:55.480043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:17:55.480048 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:17:55.481852 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:55.482399 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:55.482403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:55.482403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:55.482469 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:17:55.482741 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:17:55.482742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:17:55.482743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:17:55.482903 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:17:55.483002 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 08:17:55.483197 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:17:55.483277 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260619 08:17:55.483397 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.475253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34571 (local address 127.2.119.195:48883)
0619 08:17:55.475326 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.475348 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 08:17:55.475425 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:17:55.475611 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:17:55.475616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:17:55.475927 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:17:55.476006 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:55.476008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:55.476879 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:55.476880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:55.477547 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:17:55.477549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:17:55.478043 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:17:55.478044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:17:55.478111 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:17:55.478588 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:17:55.478593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:17:55.479937 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:17:55.482063 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:55.482071 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:55.482074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:55.482285 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:55.482589 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:17:55.482590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:17:55.482590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:17:55.482657 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:17:55.482981 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:17:55.482988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:17:55.483060 (+ 72us) client_negotiation.cc:769] Sending connection context
0619 08:17:55.483147 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:17:55.483273 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 08:17:55.483855 19122 tablet_copy_service.cc:140] P f3f8b95bef3c4b10881002c4e3d489a2: Received BeginTabletCopySession request for tablet 5fa9c10a4de94fdaa95087690194c5b1 from peer 915d4aa7921c48d5911715fc6c8c6686 ({username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:48883)
I20260619 08:17:55.483917 19122 tablet_copy_service.cc:161] P f3f8b95bef3c4b10881002c4e3d489a2: Beginning new tablet copy session on tablet 5fa9c10a4de94fdaa95087690194c5b1 from peer 915d4aa7921c48d5911715fc6c8c6686 at {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:48883: session id = 915d4aa7921c48d5911715fc6c8c6686-5fa9c10a4de94fdaa95087690194c5b1
I20260619 08:17:55.484357 19122 tablet_copy_source_session.cc:205] T 5fa9c10a4de94fdaa95087690194c5b1 P f3f8b95bef3c4b10881002c4e3d489a2: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 08:17:55.485461 19436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa9c10a4de94fdaa95087690194c5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:17:55.488050 19436 tablet_copy_client.cc:791] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: tablet copy: Starting download of 0 data blocks...
I20260619 08:17:55.488214 19436 tablet_copy_client.cc:655] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: tablet copy: Starting download of 1 WAL segments...
I20260619 08:17:55.489216 19436 tablet_copy_client.cc:531] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 08:17:55.489979 19436 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap starting.
I20260619 08:17:55.491262 19436 log.cc:824] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Log is configured to *not* fsync() on all Append() calls
I20260619 08:17:55.493662 19436 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 08:17:55.493906 19436 tablet_bootstrap.cc:492] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Bootstrap complete.
I20260619 08:17:55.494062 19436 ts_tablet_manager.cc:1397] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260619 08:17:55.494982 19436 raft_consensus.cc:348] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:55.495162 19436 raft_consensus.cc:729] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915d4aa7921c48d5911715fc6c8c6686, State: Initialized, Role: FOLLOWER
I20260619 08:17:55.495292 19436 consensus_queue.cc:260] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d4aa7921c48d5911715fc6c8c6686" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 33755 } } peers { permanent_uuid: "f3f8b95bef3c4b10881002c4e3d489a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34571 } } peers { permanent_uuid: "5c39b3934c1f4d3dbf178dff3039ce0a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35491 } }
I20260619 08:17:55.495563 19436 ts_tablet_manager.cc:1428] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 08:17:55.495919 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.495773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:44079)
0619 08:17:55.495847 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.495884 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:17:55.496033 19122 tablet_copy_service.cc:342] P f3f8b95bef3c4b10881002c4e3d489a2: Request end of tablet copy session 915d4aa7921c48d5911715fc6c8c6686-5fa9c10a4de94fdaa95087690194c5b1 received from {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:48883
I20260619 08:17:55.496088 19122 tablet_copy_service.cc:433] P f3f8b95bef3c4b10881002c4e3d489a2: ending tablet copy session 915d4aa7921c48d5911715fc6c8c6686-5fa9c10a4de94fdaa95087690194c5b1 on tablet 5fa9c10a4de94fdaa95087690194c5b1 with peer 915d4aa7921c48d5911715fc6c8c6686
I20260619 08:17:55.496286 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.496189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:43133)
0619 08:17:55.496235 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.496260 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:17:55.992566 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.992371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:41533)
0619 08:17:55.992476 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.992519 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":46,"threads_started":1}
I20260619 08:17:55.992461 19371 raft_consensus.cc:1203] T 5fa9c10a4de94fdaa95087690194c5b1 P 915d4aa7921c48d5911715fc6c8c6686 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260619 08:17:55.992961 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:55.992837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:48271)
0619 08:17:55.992899 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:55.992927 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 08:17:56.022588 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:56.022376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:51863)
0619 08:17:56.022499 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:56.022540 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260619 08:17:56.993456 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:56.993182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:36405)
0619 08:17:56.993369 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:56.993409 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":89,"threads_started":1}
I20260619 08:17:56.993616 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:56.993384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:51405)
0619 08:17:56.993499 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:56.993540 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260619 08:17:57.023389 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:57.023138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:60523)
0619 08:17:57.023277 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:57.023320 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260619 08:17:57.994230 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:57.993971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:36905)
0619 08:17:57.994137 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:57.994179 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260619 08:17:57.994287 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:57.994084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:37345)
0619 08:17:57.994211 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:57.994247 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260619 08:17:58.024144 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:58.023901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:38021)
0619 08:17:58.024068 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:58.024099 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260619 08:17:58.994997 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:58.994770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:52499)
0619 08:17:58.994914 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:58.994957 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260619 08:17:58.994990 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:58.994770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:38657)
0619 08:17:58.994909 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:58.994945 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260619 08:17:58.995509 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:58.995418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:56509)
0619 08:17:58.995463 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:58.995490 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:17:59.024982 19452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:59.024686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:57817)
0619 08:17:59.024832 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:59.024879 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260619 08:17:59.995884 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:59.995537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:47779)
0619 08:17:59.995715 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:59.995791 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":87,"threads_started":1}
I20260619 08:17:59.996176 19454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:17:59.995961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:45265)
0619 08:17:59.996098 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 08:17:59.996142 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260619 08:18:00.025786 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:00.025570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:59555)
0619 08:18:00.025695 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:00.025739 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260619 08:18:00.996752 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:00.996496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:60757)
0619 08:18:00.996656 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:00.996702 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260619 08:18:00.996752 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:00.996563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:56415)
0619 08:18:00.996676 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:00.996709 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":45,"threads_started":1}
I20260619 08:18:01.026600 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:01.026346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:43701)
0619 08:18:01.026506 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:01.026551 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260619 08:18:01.997669 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:01.997430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:59715)
0619 08:18:01.997593 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:01.997627 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
I20260619 08:18:01.997660 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:01.997430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:55885)
0619 08:18:01.997564 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:01.997610 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260619 08:18:02.027305 19461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:02.027080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:46641)
0619 08:18:02.027222 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:02.027257 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 08:18:02.998437 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:02.998215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:56965)
0619 08:18:02.998355 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:02.998388 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260619 08:18:02.998452 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:02.998212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:35799)
0619 08:18:02.998365 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:02.998409 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260619 08:18:03.028090 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:03.027858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:55469)
0619 08:18:03.027993 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:03.028036 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260619 08:18:03.999243 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:03.999006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:33107)
0619 08:18:03.999152 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:03.999194 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260619 08:18:03.999284 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:03.999016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:36361)
0619 08:18:03.999176 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:03.999221 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260619 08:18:04.028949 19467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:04.028704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:50889)
0619 08:18:04.028850 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:04.028894 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260619 08:18:05.000109 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:04.999873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:58157)
0619 08:18:05.000023 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:05.000063 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260619 08:18:05.000109 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:04.999901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:40693)
0619 08:18:05.000025 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:05.000063 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260619 08:18:05.029611 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:05.029425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:37605)
0619 08:18:05.029537 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:05.029563 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":51,"threads_started":1}
I20260619 08:18:06.000831 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:06.000607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:58633)
0619 08:18:06.000752 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:06.000785 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260619 08:18:06.000912 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:06.000605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:34593)
0619 08:18:06.000759 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:06.000857 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260619 08:18:06.030481 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:06.030209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:41321)
0619 08:18:06.030389 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:06.030432 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":62,"threads_started":1}
I20260619 08:18:07.001642 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:07.001428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:54537)
0619 08:18:07.001562 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:07.001601 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260619 08:18:07.001638 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:07.001441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:33723)
0619 08:18:07.001563 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:07.001592 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
I20260619 08:18:07.031188 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:07.030971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:60781)
0619 08:18:07.031102 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:07.031138 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":56,"threads_started":1}
I20260619 08:18:08.002380 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:08.002145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.193:56781)
0619 08:18:08.002286 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:08.002324 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260619 08:18:08.002380 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:08.002145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.195:51209)
0619 08:18:08.002286 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:08.002324 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260619 08:18:08.031977 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:08.031738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:38433 (local address 127.2.119.194:54151)
0619 08:18:08.031894 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:08.031930 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.254:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260619 08:18:08.382440 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 18985
I20260619 08:18:08.387442 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19152
I20260619 08:18:08.392050 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19288
2026-06-19T08:18:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58226 ms)
[----------] 30 tests from SecurityITest (232302 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:08 dist-test-slave-d0b5 krb5kdc[19483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:08 dist-test-slave-d0b5 krb5kdc[19483](info): set up 2 sockets
Jun 19 08:18:08 dist-test-slave-d0b5 krb5kdc[19483](info): commencing operation
krb5kdc: starting...
W20260619 08:18:09.478199 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:09 dist-test-slave-d0b5 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:09.601644 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:32865
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:32865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:09.683544 19496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:09.683759 19496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:09.683781 19496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:09.684939 19496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:09.684980 19496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:09.684995 19496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:09.685007 19496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:09.685019 19496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:09.686623 19496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:32865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:09.686923 19496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:09.687186 19496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:09.692401 19502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:09.692487 19496 server_base.cc:1034] running on GCE node
W20260619 08:18:09.692454 19504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:09.692457 19501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:09.692931 19496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:09.693277 19496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:09.694509 19496 hybrid_clock.cc:648] HybridClock initialized: now 1781857089694486 us; error 55 us; skew 500 ppm
Jun 19 08:18:09 dist-test-slave-d0b5 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:09.697155 19496 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:09.697652 19496 webserver.cc:460] Webserver started at http://127.2.119.254:43929/ using document root <none> and password file <none>
I20260619 08:18:09.697897 19496 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:09.697973 19496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:09.698130 19496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:09.699178 19496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "549e468709a4432fbf5333a5021c3895"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "b29e1a42774125fd1704c7acb11f32d0"
server_key_iv: "c291a7ce234bded428b1e49f6b2fa767"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.699540 19496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "549e468709a4432fbf5333a5021c3895"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "b29e1a42774125fd1704c7acb11f32d0"
server_key_iv: "c291a7ce234bded428b1e49f6b2fa767"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.700927 19496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:09.701885 19511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:09.702112 19496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:09.702212 19496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "549e468709a4432fbf5333a5021c3895"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "b29e1a42774125fd1704c7acb11f32d0"
server_key_iv: "c291a7ce234bded428b1e49f6b2fa767"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.702297 19496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:09.731930 19496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:09.734494 19496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:09.734651 19496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:09.738770 19496 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:32865
I20260619 08:18:09.738811 19563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:32865 every 8 connection(s)
I20260619 08:18:09.739200 19496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:09.740069 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:09.742457 19564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: Bootstrap starting.
I20260619 08:18:09.743046 19564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:09.743350 19564 log.cc:824] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:09.744107 19564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: No bootstrap required, opened a new log
I20260619 08:18:09.745927 19564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } }
I20260619 08:18:09.746068 19564 raft_consensus.cc:374] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:09.746091 19564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 549e468709a4432fbf5333a5021c3895, State: Initialized, Role: FOLLOWER
I20260619 08:18:09.746272 19564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } }
I20260619 08:18:09.746335 19564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:09.746380 19564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:09.746429 19564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:09.746837 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19496
I20260619 08:18:09.746968 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:09.747004 19564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } }
I20260619 08:18:09.747112 19564 leader_election.cc:302] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 549e468709a4432fbf5333a5021c3895; no voters:
I20260619 08:18:09.747217 2527 external_mini_cluster.cc:1428] Setting key 98b430685d6b0fd73d2eed869b3518fa
I20260619 08:18:09.747292 19564 leader_election.cc:288] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:09.747325 19567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:09.747524 19567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [term 1 LEADER]: Becoming Leader. State: Replica: 549e468709a4432fbf5333a5021c3895, State: Running, Role: LEADER
I20260619 08:18:09.747531 19564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:09.747638 19567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } }
I20260619 08:18:09.747968 19567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 549e468709a4432fbf5333a5021c3895. Latest consensus state: current_term: 1 leader_uuid: "549e468709a4432fbf5333a5021c3895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } } }
I20260619 08:18:09.748132 19567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:09.748170 19568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "549e468709a4432fbf5333a5021c3895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "549e468709a4432fbf5333a5021c3895" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 32865 } } }
I20260619 08:18:09.748265 19568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:09.748468 19578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:09.749006 19578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:09.752664 19578 catalog_manager.cc:1347] Generated new cluster ID: f8a90fce881443ac9a8ea50e25bd806f
I20260619 08:18:09.752718 19578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:18:09 dist-test-slave-d0b5 krb5kdc[19483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:09.757694 19578 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:09.758392 19578 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:09.759835 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:09.747837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.2.119.254:32865)
0619 08:18:09.748343 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:09.748352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:09.748386 (+ 34us) server_negotiation.cc:407] Connection header received
0619 08:18:09.748503 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:09.748509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:09.748583 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:09.750020 (+ 1437us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:09.750113 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.751452 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.752470 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.753142 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.753265 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:09.756104 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:09.756108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:09.756108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:09.756121 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:09.758229 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:09.758645 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:09.758647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:09.758648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:09.758748 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:09.758984 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:09.758985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:09.758985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:09.759233 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:09.759395 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:09.759493 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 08:18:09.759631 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":76,"threads_started":1}
I20260619 08:18:09.768239 19578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: Generated new TSK 0
I20260619 08:18:09.768419 19578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:18:09.801061 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:09.895002 19590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:09.895195 19590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:09.895219 19590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:09.896464 19590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:09.896507 19590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:09.896533 19590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:09.898078 19590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:09.898312 19590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:09.898540 19590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:09.903604 19596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:09.903620 19595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:09.903630 19598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:09.903892 19590 server_base.cc:1034] running on GCE node
I20260619 08:18:09.904079 19590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:09.904325 19590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:09.905498 19590 hybrid_clock.cc:648] HybridClock initialized: now 1781857089905483 us; error 26 us; skew 500 ppm
Jun 19 08:18:09 dist-test-slave-d0b5 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:09.908011 19590 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:18:09.908376 19590 webserver.cc:460] Webserver started at http://127.2.119.193:35761/ using document root <none> and password file <none>
I20260619 08:18:09.908557 19590 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:09.908597 19590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:09.908684 19590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:09.909736 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "0ec01e56cdde1229f46a8facff5488d7"
server_key_iv: "7e11fa965b2475a2d274b8b3f96cf6bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.910035 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "0ec01e56cdde1229f46a8facff5488d7"
server_key_iv: "7e11fa965b2475a2d274b8b3f96cf6bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.911254 19590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:09.911960 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:09.912124 19590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:09.912212 19590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15"
format_stamp: "Formatted at 2026-06-19 08:18:09 on dist-test-slave-d0b5"
server_key: "0ec01e56cdde1229f46a8facff5488d7"
server_key_iv: "7e11fa965b2475a2d274b8b3f96cf6bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:09.912283 19590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:09.924229 19590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:09.926167 19590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:09.926321 19590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:09.926568 19590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:09.926880 19590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:09.926942 19590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:09.926985 19590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:09.927013 19590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:09.932677 19590 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:44733
I20260619 08:18:09.932739 19718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:44733 every 8 connection(s)
I20260619 08:18:09.933068 19590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:18:09.936713 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19590
I20260619 08:18:09.936822 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:09.937032 2527 external_mini_cluster.cc:1428] Setting key 24ea347ce7f43803de40a586d57ea2fd
Jun 19 08:18:09 dist-test-slave-d0b5 krb5kdc[19483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857089, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:09.943310 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:09.933872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:60583 (local address 127.2.119.254:32865)
0619 08:18:09.933923 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:09.933925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:09.934148 (+ 223us) server_negotiation.cc:407] Connection header received
0619 08:18:09.934610 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:09.934611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:09.934622 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:09.934671 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:09.935659 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.936258 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.937711 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.938022 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.938065 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:09.940405 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:09.940407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:09.940407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:09.940415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:09.941878 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:09.942305 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:09.942306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:09.942307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:09.942370 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:09.942633 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:09.942633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:09.942634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:09.942750 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:09.942829 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:09.943131 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 08:18:09.943219 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:18:09.943516 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:09.933842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:32865 (local address 127.2.119.193:60583)
0619 08:18:09.934019 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:09.934056 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:09.934520 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:09.934730 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:09.934733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:09.935350 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:09.935553 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:09.935558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.936343 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.936344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:09.937610 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:09.937613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:09.938156 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:09.938157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:09.938298 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:09.938839 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:09.938847 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:09.940306 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:09.941989 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:09.941997 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:09.941999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:09.942226 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:09.942441 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:09.942442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:09.942442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:09.942557 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:09.942819 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:09.942825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:09.942984 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 08:18:09.943093 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 08:18:09.943251 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260619 08:18:09.944198 19719 heartbeater.cc:347] Connected to a master server at 127.2.119.254:32865
I20260619 08:18:09.944283 19719 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:09.944479 19719 heartbeater.cc:511] Master 127.2.119.254:32865 requested a full tablet report, sending...
I20260619 08:18:09.945076 19528 ts_manager.cc:194] Registered new tserver with Master: 4c8d48c0844741d1bd28ce7a9c9bbf15 (127.2.119.193:44733)
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:18:09.945983 19528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:60583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:18:09.963402 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:10.043509 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.043708 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.043731 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.044801 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.044835 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.044860 19724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:10.046242 19724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.046423 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.046590 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.051719 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.051726 19732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.051994 19724 server_base.cc:1034] running on GCE node
W20260619 08:18:10.051726 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.052237 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.052471 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.053634 19724 hybrid_clock.cc:648] HybridClock initialized: now 1781857090053630 us; error 35 us; skew 500 ppm
Jun 19 08:18:10 dist-test-slave-d0b5 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:10.056154 19724 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:18:10.056524 19724 webserver.cc:460] Webserver started at http://127.2.119.194:42715/ using document root <none> and password file <none>
I20260619 08:18:10.056692 19724 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.056731 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.056825 19724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.057871 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "f3e42287e99445be865a486283edd346"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "df7e66cbc0b3b706acccc8e5d5432ea1"
server_key_iv: "f2a30f4fc047acf102ac2d352bbe693c"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.058205 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "f3e42287e99445be865a486283edd346"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "df7e66cbc0b3b706acccc8e5d5432ea1"
server_key_iv: "f2a30f4fc047acf102ac2d352bbe693c"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.059499 19724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:10.060190 19739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.060402 19724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:10.060488 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "f3e42287e99445be865a486283edd346"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "df7e66cbc0b3b706acccc8e5d5432ea1"
server_key_iv: "f2a30f4fc047acf102ac2d352bbe693c"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.060539 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:10.075805 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:10.078224 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:10.078346 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:10.078524 19724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:10.078838 19724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:10.078873 19724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.078895 19724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:10.078908 19724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.084683 19724 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:36383
I20260619 08:18:10.084729 19852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:36383 every 8 connection(s)
I20260619 08:18:10.085039 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:18:10.088028 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19724
I20260619 08:18:10.088148 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:10.088366 2527 external_mini_cluster.cc:1428] Setting key f5544ce1ea999d2c86e6e2cfff69048b
Jun 19 08:18:10 dist-test-slave-d0b5 krb5kdc[19483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:10.095222 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.085744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:34567 (local address 127.2.119.254:32865)
0619 08:18:10.085815 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.085818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.086033 (+ 215us) server_negotiation.cc:407] Connection header received
0619 08:18:10.086533 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.086534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.086544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.086605 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.087564 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.088054 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.089579 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.089777 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.089856 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.092260 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.092262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.092262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:10.092270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.093804 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.094237 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.094239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.094239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.094302 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.094566 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.094567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.094567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.094687 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.094763 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.095048 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.095129 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:18:10.095422 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.085727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:32865 (local address 127.2.119.194:34567)
0619 08:18:10.085900 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.085947 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.086407 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.086678 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.086682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.087263 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:10.087472 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.087477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.088173 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.088174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.089476 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.089483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.089862 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.089864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.090003 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.090532 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:10.090547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:10.092158 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:10.093909 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:10.093917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:10.093919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:10.094157 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:10.094377 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:10.094378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:10.094378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:10.094493 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:10.094753 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:10.094760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:10.094905 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.095017 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.095186 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260619 08:18:10.095964 19853 heartbeater.cc:347] Connected to a master server at 127.2.119.254:32865
I20260619 08:18:10.096076 19853 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:10.096272 19853 heartbeater.cc:511] Master 127.2.119.254:32865 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
I20260619 08:18:10.096702 19528 ts_manager.cc:194] Registered new tserver with Master: f3e42287e99445be865a486283edd346 (127.2.119.194:36383)
I20260619 08:18:10.097095 19528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:34567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:18:10.116338 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:10.196966 19858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.197162 19858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.197185 19858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.198354 19858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.198395 19858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.198421 19858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:10.199761 19858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.200016 19858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.200259 19858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.205384 19864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.205436 19863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.205416 19866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.205538 19858 server_base.cc:1034] running on GCE node
I20260619 08:18:10.205746 19858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.206020 19858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.207165 19858 hybrid_clock.cc:648] HybridClock initialized: now 1781857090207153 us; error 29 us; skew 500 ppm
Jun 19 08:18:10 dist-test-slave-d0b5 krb5kdc[19483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:10.209764 19858 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:18:10.210199 19858 webserver.cc:460] Webserver started at http://127.2.119.195:39699/ using document root <none> and password file <none>
I20260619 08:18:10.210435 19858 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.210511 19858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.210668 19858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.211761 19858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "9413f19dd4614c6489d89db9f4712e47"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6d61439285080515c5a57672bf6a6520"
server_key_iv: "61a8c705be67998a3101ce67613cfa81"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.212131 19858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "9413f19dd4614c6489d89db9f4712e47"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6d61439285080515c5a57672bf6a6520"
server_key_iv: "61a8c705be67998a3101ce67613cfa81"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.213558 19858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 08:18:10.214293 19873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.214479 19858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:10.214586 19858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "9413f19dd4614c6489d89db9f4712e47"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6d61439285080515c5a57672bf6a6520"
server_key_iv: "61a8c705be67998a3101ce67613cfa81"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.214659 19858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:10.225660 19858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:10.227866 19858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:10.228018 19858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:10.228223 19858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:10.228516 19858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:10.228570 19858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.228611 19858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:10.228638 19858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.234511 19858 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:43945
I20260619 08:18:10.234548 19986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:43945 every 8 connection(s)
I20260619 08:18:10.234860 19858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:18:10 dist-test-slave-d0b5 krb5kdc[19483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857090, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:10.241254 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 19858
I20260619 08:18:10.241400 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:10.241631 2527 external_mini_cluster.cc:1428] Setting key 474b69b8af222f3fef8f5c5895404f0a
I20260619 08:18:10.244485 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.235597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:55521 (local address 127.2.119.254:32865)
0619 08:18:10.235668 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.235671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.235901 (+ 230us) server_negotiation.cc:407] Connection header received
0619 08:18:10.236362 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.236363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.236373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.236430 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.237353 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.237865 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.238893 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.239110 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.239169 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.241599 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.241600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.241601 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:10.241608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.243172 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.243537 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.243538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.243538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.243586 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.243887 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.243889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.243889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.244020 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.244085 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.244338 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.244421 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:18:10.244613 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.235551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:32865 (local address 127.2.119.195:55521)
0619 08:18:10.235752 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.235796 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.236264 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.236478 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.236482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.237032 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:10.237216 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.237220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.237979 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.237980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.238788 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.238790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.239189 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.239190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.239287 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.239809 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:10.239833 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:10.241438 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:10.243245 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:10.243251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:10.243253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:10.243473 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:10.243644 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:10.243644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:10.243645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:10.243740 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:10.244094 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:10.244099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:10.244213 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.244300 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.244423 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260619 08:18:10.245095 19987 heartbeater.cc:347] Connected to a master server at 127.2.119.254:32865
I20260619 08:18:10.245186 19987 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:10.245414 19987 heartbeater.cc:511] Master 127.2.119.254:32865 requested a full tablet report, sending...
I20260619 08:18:10.245786 19528 ts_manager.cc:194] Registered new tserver with Master: 9413f19dd4614c6489d89db9f4712e47 (127.2.119.195:43945)
I20260619 08:18:10.246201 19528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:55521
I20260619 08:18:10.253433 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:10.261270 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.254885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.2.119.254:32865)
0619 08:18:10.254949 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.254951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.255005 (+ 54us) server_negotiation.cc:407] Connection header received
0619 08:18:10.255082 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.255083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.255093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.255164 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.255765 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.256259 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.256986 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.257177 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.257277 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.258518 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.258519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.258519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:10.258527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.259751 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.260275 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.260276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.260277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.260358 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:10.260602 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:10.260603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:10.260604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:10.260857 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.260951 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.261077 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.261156 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:18:10.263077 19528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:10.264848 19528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:10.270828 19995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.266903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44733 (local address 127.0.0.1:52434)
0619 08:18:10.267256 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.267266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.267307 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.267597 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.267598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.267600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:10.267651 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.267652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.268542 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.268543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.269623 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.269626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.270519 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.270600 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.270613 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.270694 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.270754 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":52,"threads_started":1}
I20260619 08:18:10.270840 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.267089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.2.119.193:44733)
0619 08:18:10.267420 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.267425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.267445 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:18:10.267496 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.267497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.267508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.267587 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:10.267712 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.268435 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.269697 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.270431 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270503 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.270561 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.270717 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.270759 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":85,"threads_started":1}
I20260619 08:18:10.271107 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.266904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:36383 (local address 127.0.0.1:43618)
0619 08:18:10.267365 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.267375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.267411 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.267814 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.267815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.267817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:10.267860 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.267862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.269106 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.269106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.270028 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.270030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.270852 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.270916 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.270920 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.270997 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.271041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":95,"threads_started":1}
I20260619 08:18:10.271154 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.267169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43618 (local address 127.2.119.194:36383)
0619 08:18:10.267599 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.267603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.267612 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:10.267649 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.267651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.267665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.267737 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:10.267938 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.268999 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270129 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.270710 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270769 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.270819 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.271018 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.271067 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":70,"threads_started":1}
I20260619 08:18:10.271765 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.266902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:43945 (local address 127.0.0.1:47666)
0619 08:18:10.267150 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.267192 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.267259 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.267487 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.267490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.267493 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:10.267563 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.267564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.269051 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.269051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.270167 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.270170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.271553 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.271553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.271600 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.271603 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.271664 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.271703 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":63,"threads_started":1}
I20260619 08:18:10.271662 19653 tablet_service.cc:1467] Processing CreateTablet for tablet 7dac7c52c7c445ccb869be67e013b06e (DEFAULT_TABLE table=test-table [id=0c0ab11cf36b4fb9963ada0e8dd69669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:10.271811 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.267038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.2.119.195:43945)
0619 08:18:10.267277 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.267282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.267293 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:10.267336 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.267338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.267362 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.267478 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:10.267656 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.268832 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.270266 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.271012 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.271060 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.271106 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.271692 (+ 586us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.271731 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":111,"threads_started":1}
I20260619 08:18:10.271900 19787 tablet_service.cc:1467] Processing CreateTablet for tablet 7dac7c52c7c445ccb869be67e013b06e (DEFAULT_TABLE table=test-table [id=0c0ab11cf36b4fb9963ada0e8dd69669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:10.272027 19653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dac7c52c7c445ccb869be67e013b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:10.272197 19787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dac7c52c7c445ccb869be67e013b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:10.272496 19921 tablet_service.cc:1467] Processing CreateTablet for tablet 7dac7c52c7c445ccb869be67e013b06e (DEFAULT_TABLE table=test-table [id=0c0ab11cf36b4fb9963ada0e8dd69669]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:10.272800 19921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dac7c52c7c445ccb869be67e013b06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:10.275346 20003 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: Bootstrap starting.
I20260619 08:18:10.275507 20001 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Bootstrap starting.
I20260619 08:18:10.276093 20003 tablet_bootstrap.cc:654] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:10.276150 20001 tablet_bootstrap.cc:654] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:10.276701 20001 log.cc:824] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:10.276727 20003 log.cc:824] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:10.277045 20002 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Bootstrap starting.
I20260619 08:18:10.277529 20003 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: No bootstrap required, opened a new log
I20260619 08:18:10.277539 20001 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: No bootstrap required, opened a new log
I20260619 08:18:10.277606 20001 ts_tablet_manager.cc:1397] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:10.277606 20003 ts_tablet_manager.cc:1397] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:10.277721 20002 tablet_bootstrap.cc:654] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:10.278257 20002 log.cc:824] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:10.278963 20002 tablet_bootstrap.cc:492] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: No bootstrap required, opened a new log
I20260619 08:18:10.279043 20002 ts_tablet_manager.cc:1397] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:18:10.279557 20001 raft_consensus.cc:348] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.279557 20003 raft_consensus.cc:348] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.279736 20001 raft_consensus.cc:374] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:10.279736 20003 raft_consensus.cc:374] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:10.279772 20003 raft_consensus.cc:729] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c8d48c0844741d1bd28ce7a9c9bbf15, State: Initialized, Role: FOLLOWER
I20260619 08:18:10.279772 20001 raft_consensus.cc:729] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9413f19dd4614c6489d89db9f4712e47, State: Initialized, Role: FOLLOWER
I20260619 08:18:10.279910 20001 consensus_queue.cc:260] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.279910 20003 consensus_queue.cc:260] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.280187 20003 ts_tablet_manager.cc:1428] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:18:10.280187 20001 ts_tablet_manager.cc:1428] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:18:10.280308 19719 heartbeater.cc:503] Master 127.2.119.254:32865 was elected leader, sending a full tablet report...
I20260619 08:18:10.280315 19987 heartbeater.cc:503] Master 127.2.119.254:32865 was elected leader, sending a full tablet report...
I20260619 08:18:10.280804 20002 raft_consensus.cc:348] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.280980 20002 raft_consensus.cc:374] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:10.281031 20002 raft_consensus.cc:729] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e42287e99445be865a486283edd346, State: Initialized, Role: FOLLOWER
I20260619 08:18:10.281137 20002 consensus_queue.cc:260] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.281426 20002 ts_tablet_manager.cc:1428] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 08:18:10.281564 19853 heartbeater.cc:503] Master 127.2.119.254:32865 was elected leader, sending a full tablet report...
W20260619 08:18:10.335662 19854 tablet.cc:2369] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:10.382506 20008 raft_consensus.cc:482] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:10.382635 20008 raft_consensus.cc:504] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.383037 20008 leader_election.cc:288] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9413f19dd4614c6489d89db9f4712e47 (127.2.119.195:43945), f3e42287e99445be865a486283edd346 (127.2.119.194:36383)
I20260619 08:18:10.387135 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.383200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:43945 (local address 127.2.119.193:33853)
0619 08:18:10.383284 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.383297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.383344 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.383525 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.383527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.383528 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:10.383605 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.383607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.384566 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.384567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.385955 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.385960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.386814 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.386910 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.386911 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.386981 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.387030 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":84}
I20260619 08:18:10.387176 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.383299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:58179 (local address 127.2.119.194:36383)
0619 08:18:10.383342 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.383345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.383507 (+ 162us) server_negotiation.cc:407] Connection header received
0619 08:18:10.383567 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.383569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.383584 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.383627 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:10.383793 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.384479 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386095 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.386705 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386790 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.386799 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.387051 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.387101 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:10.387176 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.383286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:33853 (local address 127.2.119.195:43945)
0619 08:18:10.383333 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.383335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.383346 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:10.383411 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.383412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.383423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.383508 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:10.383702 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.384470 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386095 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.386705 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386789 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.386799 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.387051 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.387101 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:18:10.387142 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.383244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:36383 (local address 127.2.119.193:58179)
0619 08:18:10.383417 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.383427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.383505 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.383653 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.383654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.383656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:10.383717 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.383718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.384566 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.384567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.386000 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.386002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.386813 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.386815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.386906 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.386911 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.387001 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.387031 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":95,"thread_start_us":109,"threads_started":1}
I20260619 08:18:10.387557 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dac7c52c7c445ccb869be67e013b06e" candidate_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9413f19dd4614c6489d89db9f4712e47" is_pre_election: true
I20260619 08:18:10.387631 19807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dac7c52c7c445ccb869be67e013b06e" candidate_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e42287e99445be865a486283edd346" is_pre_election: true
I20260619 08:18:10.387739 19941 raft_consensus.cc:2454] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c8d48c0844741d1bd28ce7a9c9bbf15 in term 0.
I20260619 08:18:10.387799 19807 raft_consensus.cc:2454] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c8d48c0844741d1bd28ce7a9c9bbf15 in term 0.
I20260619 08:18:10.387986 19607 leader_election.cc:302] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c8d48c0844741d1bd28ce7a9c9bbf15, 9413f19dd4614c6489d89db9f4712e47; no voters:
I20260619 08:18:10.388172 20008 raft_consensus.cc:2781] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:10.388231 20008 raft_consensus.cc:482] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:10.388255 20008 raft_consensus.cc:3037] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:10.388960 20008 raft_consensus.cc:504] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.389106 20008 leader_election.cc:288] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [CANDIDATE]: Term 1 election: Requested vote from peers 9413f19dd4614c6489d89db9f4712e47 (127.2.119.195:43945), f3e42287e99445be865a486283edd346 (127.2.119.194:36383)
I20260619 08:18:10.389245 19941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dac7c52c7c445ccb869be67e013b06e" candidate_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9413f19dd4614c6489d89db9f4712e47"
I20260619 08:18:10.389253 19807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dac7c52c7c445ccb869be67e013b06e" candidate_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e42287e99445be865a486283edd346"
I20260619 08:18:10.389364 19807 raft_consensus.cc:3037] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:10.389365 19941 raft_consensus.cc:3037] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:10.390064 19807 raft_consensus.cc:2454] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c8d48c0844741d1bd28ce7a9c9bbf15 in term 1.
I20260619 08:18:10.390262 19941 raft_consensus.cc:2454] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c8d48c0844741d1bd28ce7a9c9bbf15 in term 1.
I20260619 08:18:10.390252 19606 leader_election.cc:302] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c8d48c0844741d1bd28ce7a9c9bbf15, f3e42287e99445be865a486283edd346; no voters:
I20260619 08:18:10.390365 20008 raft_consensus.cc:2781] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:10.390534 20008 raft_consensus.cc:686] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [term 1 LEADER]: Becoming Leader. State: Replica: 4c8d48c0844741d1bd28ce7a9c9bbf15, State: Running, Role: LEADER
I20260619 08:18:10.390607 20008 consensus_queue.cc:237] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } }
I20260619 08:18:10.391433 19528 catalog_manager.cc:5495] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c8d48c0844741d1bd28ce7a9c9bbf15 (127.2.119.193). New cstate: current_term: 1 leader_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c8d48c0844741d1bd28ce7a9c9bbf15" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 44733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 } health_report { overall_health: UNKNOWN } } }
I20260619 08:18:10.421836 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.419020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.2.119.193:44733)
0619 08:18:10.419061 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.419064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.419073 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:10.419164 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.419165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.419193 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.419329 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:10.419508 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.420036 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.420949 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.421180 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.421276 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.421486 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:10.421588 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:10.421693 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.421720 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.421741 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:18:10.423717 19941 raft_consensus.cc:1261] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 1 FOLLOWER]: Refusing update from remote peer 4c8d48c0844741d1bd28ce7a9c9bbf15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:10.423795 19807 raft_consensus.cc:1261] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 1 FOLLOWER]: Refusing update from remote peer 4c8d48c0844741d1bd28ce7a9c9bbf15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:10.424165 20008 consensus_queue.cc:1035] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e42287e99445be865a486283edd346" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:10.424316 20011 consensus_queue.cc:1035] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9413f19dd4614c6489d89db9f4712e47" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 43945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:10.429014 19524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:10.429116 19524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:10.430008 19524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 549e468709a4432fbf5333a5021c3895: Sending DeleteTablet for 3 replicas of tablet 7dac7c52c7c445ccb869be67e013b06e
I20260619 08:18:10.430331 19921 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dac7c52c7c445ccb869be67e013b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:10 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:47666
I20260619 08:18:10.430346 19653 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dac7c52c7c445ccb869be67e013b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:10 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:52434
I20260619 08:18:10.430416 19787 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dac7c52c7c445ccb869be67e013b06e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:10 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:43618
I20260619 08:18:10.430601 20026 tablet_replica.cc:330] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: stopping tablet replica
I20260619 08:18:10.430706 20026 raft_consensus.cc:2229] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:10.430814 20026 raft_consensus.cc:2258] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:10.430922 20025 tablet_replica.cc:330] T 7dac7c52c7c445ccb869be67e013b06e P 4c8d48c0844741d1bd28ce7a9c9bbf15: stopping tablet replica
I20260619 08:18:10.430948 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19590
I20260619 08:18:10.430842 20024 tablet_replica.cc:330] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: stopping tablet replica
I20260619 08:18:10.431819 20024 raft_consensus.cc:2229] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:10.431847 20026 ts_tablet_manager.cc:1905] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:10.431984 20024 raft_consensus.cc:2258] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:10.432375 20024 ts_tablet_manager.cc:1905] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:10.433883 20026 ts_tablet_manager.cc:1918] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:10.433966 20026 log.cc:1196] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/7dac7c52c7c445ccb869be67e013b06e
I20260619 08:18:10.434258 20026 ts_tablet_manager.cc:1939] T 7dac7c52c7c445ccb869be67e013b06e P f3e42287e99445be865a486283edd346: Deleting consensus metadata
I20260619 08:18:10.434334 20024 ts_tablet_manager.cc:1918] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:10.434401 20024 log.cc:1196] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/7dac7c52c7c445ccb869be67e013b06e
I20260619 08:18:10.434659 20024 ts_tablet_manager.cc:1939] T 7dac7c52c7c445ccb869be67e013b06e P 9413f19dd4614c6489d89db9f4712e47: Deleting consensus metadata
I20260619 08:18:10.434680 19512 catalog_manager.cc:4841] TS f3e42287e99445be865a486283edd346 (127.2.119.194:36383): tablet 7dac7c52c7c445ccb869be67e013b06e (table test-table [id=0c0ab11cf36b4fb9963ada0e8dd69669]) successfully deleted
I20260619 08:18:10.435034 19513 catalog_manager.cc:4841] TS 9413f19dd4614c6489d89db9f4712e47 (127.2.119.195:43945): tablet 7dac7c52c7c445ccb869be67e013b06e (table test-table [id=0c0ab11cf36b4fb9963ada0e8dd69669]) successfully deleted
W20260619 08:18:10.437273 19515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:44733 (error 108)
I20260619 08:18:10.437517 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19724
I20260619 08:18:10.438143 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.437650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:44733 (local address 127.0.0.1:52454)
0619 08:18:10.438066 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.438102 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396}
W20260619 08:18:10.438453 19515 catalog_manager.cc:4584] TS 4c8d48c0844741d1bd28ce7a9c9bbf15 (127.2.119.193:44733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7dac7c52c7c445ccb869be67e013b06e: Network error: Client connection negotiation failed: client connection to 127.2.119.193:44733: connect: Connection refused (error 111)
I20260619 08:18:10.442365 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19858
I20260619 08:18:10.447072 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 19496
2026-06-19T08:18:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2055 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T08:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:10Z Disabled control of system clock
I20260619 08:18:10.475422 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:37017
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:37017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:10.557214 20031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.557458 20031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.557480 20031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.558579 20031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:10.558612 20031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.558625 20031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.558638 20031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:10.558650 20031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:10.559898 20031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:37017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:37017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.560096 20031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.560343 20031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.565268 20039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.565256 20037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.565505 20031 server_base.cc:1034] running on GCE node
W20260619 08:18:10.565253 20036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.566135 20031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.566454 20031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.567642 20031 hybrid_clock.cc:648] HybridClock initialized: now 1781857090567613 us; error 50 us; skew 500 ppm
I20260619 08:18:10.568888 20031 webserver.cc:460] Webserver started at http://127.2.119.254:45283/ using document root <none> and password file <none>
I20260619 08:18:10.569140 20031 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.569212 20031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.569396 20031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.570531 20031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "5091d4d7a63641b98c3b901a89fb973b"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "a6df37971e167efd6ebe2490b872edd3"
server_key_iv: "bd2a606133b5bdc0b5e511a6eb06f3ae"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.570896 20031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "5091d4d7a63641b98c3b901a89fb973b"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "a6df37971e167efd6ebe2490b872edd3"
server_key_iv: "bd2a606133b5bdc0b5e511a6eb06f3ae"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.572219 20031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:18:10.573063 20045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.573390 20031 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:10.573526 20031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "5091d4d7a63641b98c3b901a89fb973b"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "a6df37971e167efd6ebe2490b872edd3"
server_key_iv: "bd2a606133b5bdc0b5e511a6eb06f3ae"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.573621 20031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:10.589133 20031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:10.589545 20031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:10.589706 20031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:10.593495 20031 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:37017
I20260619 08:18:10.593556 20097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:37017 every 8 connection(s)
I20260619 08:18:10.593922 20031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:10.594996 20098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:10.597652 20098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: Bootstrap starting.
I20260619 08:18:10.598327 20098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:10.598663 20098 log.cc:824] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:10.599422 20098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: No bootstrap required, opened a new log
I20260619 08:18:10.600059 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20031
I20260619 08:18:10.600189 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:10.600406 2527 external_mini_cluster.cc:1428] Setting key 8cf51dbd343c54d744940eba9258c7f9
I20260619 08:18:10.601989 20098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } }
I20260619 08:18:10.602133 20098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:10.602157 20098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5091d4d7a63641b98c3b901a89fb973b, State: Initialized, Role: FOLLOWER
I20260619 08:18:10.602324 20098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } }
I20260619 08:18:10.602417 20098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:10.602458 20098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:10.602511 20098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:10.603128 20098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } }
I20260619 08:18:10.603221 20098 leader_election.cc:302] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5091d4d7a63641b98c3b901a89fb973b; no voters:
I20260619 08:18:10.603505 20098 leader_election.cc:288] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:10.603539 20103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:10.603654 20103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [term 1 LEADER]: Becoming Leader. State: Replica: 5091d4d7a63641b98c3b901a89fb973b, State: Running, Role: LEADER
I20260619 08:18:10.603825 20098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:10.603816 20103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } }
I20260619 08:18:10.604308 20104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5091d4d7a63641b98c3b901a89fb973b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } } }
I20260619 08:18:10.604606 20104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:10.604499 20105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5091d4d7a63641b98c3b901a89fb973b. Latest consensus state: current_term: 1 leader_uuid: "5091d4d7a63641b98c3b901a89fb973b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5091d4d7a63641b98c3b901a89fb973b" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 37017 } } }
I20260619 08:18:10.605063 20112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:10.605665 20112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:10.605095 20105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:10.607308 20112 catalog_manager.cc:1347] Generated new cluster ID: 8f6d25ac91b34326834d25d5210dfa06
I20260619 08:18:10.607388 20112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:10.607831 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.601071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.2.119.254:37017)
0619 08:18:10.601341 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.601350 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.601389 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:18:10.601983 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.601990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.602059 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.602316 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.602603 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.604109 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.605195 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.605834 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.605959 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.606983 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.606987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.606987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:10.607161 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.607193 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:10.607228 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.607304 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.607568 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.607659 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260619 08:18:10.617446 20112 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:10.618153 20112 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:10.627146 20112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: Generated new TSK 0
I20260619 08:18:10.627337 20112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:10.628435 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--builtin_ntp_servers=127.2.119.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:10.715055 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.715296 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.715330 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.716490 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.716557 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.716600 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:10.718025 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.718277 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.718568 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.725010 20127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.724970 20128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.724949 20130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.725184 20122 server_base.cc:1034] running on GCE node
I20260619 08:18:10.725487 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.725811 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.727011 20122 hybrid_clock.cc:648] HybridClock initialized: now 1781857090727002 us; error 33 us; skew 500 ppm
I20260619 08:18:10.728643 20122 webserver.cc:460] Webserver started at http://127.2.119.193:43643/ using document root <none> and password file <none>
I20260619 08:18:10.728935 20122 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.729028 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.729220 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.730909 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "0704e2d8a88a77da532e667b0f3c2f8a"
server_key_iv: "947fd6fa4c2ecc0f1cbd4b0eb99bccef"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.731367 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "0704e2d8a88a77da532e667b0f3c2f8a"
server_key_iv: "947fd6fa4c2ecc0f1cbd4b0eb99bccef"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.733387 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 08:18:10.734535 20136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.734750 20122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:10.734881 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "0704e2d8a88a77da532e667b0f3c2f8a"
server_key_iv: "947fd6fa4c2ecc0f1cbd4b0eb99bccef"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.734979 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:10.750831 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:10.751147 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:10.751303 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:10.751533 20122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:10.751864 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:10.751924 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.751964 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:10.751992 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.757705 20122 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:41609
I20260619 08:18:10.757751 20249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:41609 every 8 connection(s)
I20260619 08:18:10.758123 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:18:10.764016 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.758891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:53963 (local address 127.2.119.254:37017)
0619 08:18:10.759140 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.759143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.759338 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:18:10.759917 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.759919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.759935 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.760000 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.760749 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.761361 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.762470 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.762692 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.762787 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.763774 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.763775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.763775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:10.763781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.763794 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:10.763798 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.763845 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.763927 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.763952 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260619 08:18:10.764204 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20122
I20260619 08:18:10.764264 20252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.758846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37017 (local address 127.2.119.193:53963)
0619 08:18:10.759199 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.759242 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.759798 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.760079 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.760083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.760103 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:10.760640 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.760645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.761473 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.761474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.762356 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.762359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.762824 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.762826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.763040 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.763638 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:10.763638 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:10.763668 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:10.763692 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:10.763848 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:10.763852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:10.763855 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.763909 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.764016 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":74,"threads_started":1}
I20260619 08:18:10.764351 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:10.764586 2527 external_mini_cluster.cc:1428] Setting key 2d2ec8f282a05df079044c51251605a0
I20260619 08:18:10.764932 20250 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37017
I20260619 08:18:10.765053 20250 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:10.765244 20250 heartbeater.cc:511] Master 127.2.119.254:37017 requested a full tablet report, sending...
I20260619 08:18:10.765735 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--builtin_ntp_servers=127.2.119.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:18:10.765985 20062 ts_manager.cc:194] Registered new tserver with Master: 7fae5bc02c5c44d0b6a27612aaf27dc5 (127.2.119.193:41609)
I20260619 08:18:10.766597 20062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.193:53963
W20260619 08:18:10.851402 20253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.851613 20253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.851635 20253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.852852 20253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.852907 20253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.852947 20253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:10.854451 20253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.854650 20253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.854894 20253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.860010 20258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.860016 20259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.860044 20261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.860049 20253 server_base.cc:1034] running on GCE node
I20260619 08:18:10.860458 20253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.860746 20253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.861886 20253 hybrid_clock.cc:648] HybridClock initialized: now 1781857090861875 us; error 26 us; skew 500 ppm
I20260619 08:18:10.863204 20253 webserver.cc:460] Webserver started at http://127.2.119.194:35153/ using document root <none> and password file <none>
I20260619 08:18:10.863471 20253 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.863521 20253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.863654 20253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.864732 20253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "ab5d789caf61441b987ad92e6e011b79"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "1b8bd744e9f205a905b79dbb0e795c10"
server_key_iv: "2e0c8a0499cc339f622597ddfa8aa25e"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.865091 20253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "ab5d789caf61441b987ad92e6e011b79"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "1b8bd744e9f205a905b79dbb0e795c10"
server_key_iv: "2e0c8a0499cc339f622597ddfa8aa25e"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.866461 20253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:10.867257 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.867455 20253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:10.867559 20253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "ab5d789caf61441b987ad92e6e011b79"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "1b8bd744e9f205a905b79dbb0e795c10"
server_key_iv: "2e0c8a0499cc339f622597ddfa8aa25e"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.867633 20253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:10.877990 20253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:10.878315 20253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:10.878474 20253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:10.878713 20253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:10.879051 20253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:10.879107 20253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.879140 20253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:10.879166 20253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.884975 20253 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:36873
I20260619 08:18:10.885023 20380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:36873 every 8 connection(s)
I20260619 08:18:10.885396 20253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:18:10.890723 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.886137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:41581 (local address 127.2.119.254:37017)
0619 08:18:10.886185 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:10.886187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:10.886423 (+ 236us) server_negotiation.cc:407] Connection header received
0619 08:18:10.886976 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:10.886978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:10.886997 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:10.887069 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:10.887766 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.888283 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.889326 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.889548 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.889631 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.890433 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:10.890434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:10.890435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:10.890440 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:10.890455 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:10.890460 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:10.890503 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:10.890616 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:18:10.890645 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:10.890774 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20253
I20260619 08:18:10.890900 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:10.891000 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:10.886093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37017 (local address 127.2.119.194:41581)
0619 08:18:10.886298 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:10.886333 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:10.886852 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:10.887149 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:10.887152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:10.887168 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:10.887657 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.887660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.888396 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.888397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.889190 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:10.889192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:10.889633 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:10.889633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:10.889771 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:10.890291 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:10.890291 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:10.890319 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:10.890340 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:10.890515 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:10.890519 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:10.890521 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:18:10.890607 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:18:10.890706 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260619 08:18:10.891121 2527 external_mini_cluster.cc:1428] Setting key 31a1fd6ec3d82f832f9db7912453763a
I20260619 08:18:10.891461 20381 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37017
I20260619 08:18:10.891552 20381 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:10.891726 20381 heartbeater.cc:511] Master 127.2.119.254:37017 requested a full tablet report, sending...
I20260619 08:18:10.892081 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--builtin_ntp_servers=127.2.119.212:39363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:18:10.892127 20062 ts_manager.cc:194] Registered new tserver with Master: ab5d789caf61441b987ad92e6e011b79 (127.2.119.194:36873)
I20260619 08:18:10.892520 20062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.194:41581
W20260619 08:18:10.971925 20384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:10.972150 20384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:10.972184 20384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:10.973367 20384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:10.973423 20384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:10.973464 20384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:10.974784 20384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:39363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:37017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:10.975039 20384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:10.975333 20384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:10.980515 20389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.980592 20384 server_base.cc:1034] running on GCE node
W20260619 08:18:10.980546 20392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:10.980557 20390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:10.980968 20384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:10.981216 20384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:10.982376 20384 hybrid_clock.cc:648] HybridClock initialized: now 1781857090982366 us; error 30 us; skew 500 ppm
I20260619 08:18:10.983608 20384 webserver.cc:460] Webserver started at http://127.2.119.195:38439/ using document root <none> and password file <none>
I20260619 08:18:10.983872 20384 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:10.983924 20384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:10.984054 20384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:10.985095 20384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "da6c15a27a5c48d59c8f7a82191c3277"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6271fc65059355c9b18abd2064e00106"
server_key_iv: "548d83c401cf06a6ef56e705b7821527"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.985499 20384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "da6c15a27a5c48d59c8f7a82191c3277"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6271fc65059355c9b18abd2064e00106"
server_key_iv: "548d83c401cf06a6ef56e705b7821527"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.986833 20384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:10.987543 20398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:10.987753 20384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:18:10.987824 20384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "da6c15a27a5c48d59c8f7a82191c3277"
format_stamp: "Formatted at 2026-06-19 08:18:10 on dist-test-slave-d0b5"
server_key: "6271fc65059355c9b18abd2064e00106"
server_key_iv: "548d83c401cf06a6ef56e705b7821527"
server_key_version: "encryptionkey@0"
I20260619 08:18:10.987910 20384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:11.004470 20384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:11.004778 20384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:11.004933 20384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:11.005174 20384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:11.005577 20384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:11.005611 20384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:11.005635 20384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:11.005681 20384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:11.011678 20384 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:34525
I20260619 08:18:11.011720 20511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:34525 every 8 connection(s)
I20260619 08:18:11.012038 20384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:18:11.017143 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20384
I20260619 08:18:11.017251 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:11.017493 2527 external_mini_cluster.cc:1428] Setting key 485bd64f2fb97fe39ba0970a4eca2b2c
I20260619 08:18:11.017616 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.012865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:34543 (local address 127.2.119.254:37017)
0619 08:18:11.012951 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.012954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.013159 (+ 205us) server_negotiation.cc:407] Connection header received
0619 08:18:11.013729 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.013730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.013741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.013805 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:11.014466 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.015047 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.016056 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.016261 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.016322 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.017203 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:11.017204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:11.017205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:11.017235 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:11.017254 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:11.017259 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:11.017350 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.017486 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.017519 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 08:18:11.017854 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.012806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:37017 (local address 127.2.119.195:34543)
0619 08:18:11.013034 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.013067 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.013632 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.013874 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.013878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.013892 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:11.014381 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.014385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.015141 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.015141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.015966 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.015969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.016353 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.016354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.016459 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.017005 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:11.017005 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:11.017047 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:11.017079 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:11.017352 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:11.017377 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:11.017380 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.017486 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.017597 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":118,"threads_started":1}
I20260619 08:18:11.018324 20512 heartbeater.cc:347] Connected to a master server at 127.2.119.254:37017
I20260619 08:18:11.018451 20512 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:11.018648 20512 heartbeater.cc:511] Master 127.2.119.254:37017 requested a full tablet report, sending...
I20260619 08:18:11.019148 20062 ts_manager.cc:194] Registered new tserver with Master: da6c15a27a5c48d59c8f7a82191c3277 (127.2.119.195:34525)
I20260619 08:18:11.019563 20062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.195:34543
I20260619 08:18:11.029791 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:11.034565 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.031590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.2.119.254:37017)
0619 08:18:11.031669 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.031671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.031733 (+ 62us) server_negotiation.cc:407] Connection header received
0619 08:18:11.031775 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.031775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.031787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.031842 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:11.032042 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.032529 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.033396 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.033586 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.033638 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.034338 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:11.034339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:11.034339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:11.034343 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:11.034356 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:11.034360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:11.034402 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.034484 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.034507 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 08:18:11.036190 20062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:11.037838 20062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:11.044960 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41609 (local address 127.0.0.1:53412)
0619 08:18:11.040427 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.040443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.040569 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.040884 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.040884 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.040886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:11.040944 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.040945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.042317 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.042318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.043756 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.043759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.044554 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.044640 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.044655 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.044743 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.044877 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":70,"thread_start_us":84,"threads_started":1}
I20260619 08:18:11.045176 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:34525 (local address 127.0.0.1:52794)
0619 08:18:11.040640 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.040651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.040706 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.040881 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.040882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.040884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:11.040942 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.040944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.043251 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.043252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.044049 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.044051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.044936 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.044997 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.045001 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.045055 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.045104 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260619 08:18:11.045181 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.2.119.193:41609)
0619 08:18:11.040364 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.040369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.040524 (+ 155us) server_negotiation.cc:407] Connection header received
0619 08:18:11.040686 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.040689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.040706 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.040793 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:11.041024 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.042197 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.043852 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.044461 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044519 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.044582 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.045045 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.045101 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260619 08:18:11.045496 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.2.119.195:34525)
0619 08:18:11.040589 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.040594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.040706 (+ 112us) server_negotiation.cc:407] Connection header received
0619 08:18:11.040784 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.040786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.040801 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.040870 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:11.042310 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.043163 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044157 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.044742 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044803 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.044888 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.045358 (+ 470us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.045411 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260619 08:18:11.046051 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:36873 (local address 127.0.0.1:44594)
0619 08:18:11.040253 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.040297 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.040365 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.040830 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.040833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.040835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:11.040900 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.040902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.042308 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.042309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.043756 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.043759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.045792 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.045793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.045857 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.045862 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.045924 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.045981 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":58,"thread_start_us":67,"threads_started":1}
I20260619 08:18:11.045925 20184 tablet_service.cc:1467] Processing CreateTablet for tablet 76f69ecff4db472ab0666dba1d88f481 (DEFAULT_TABLE table=test-table [id=aa4815c879d04cc1874bc3a5e18be2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:11.046053 20446 tablet_service.cc:1467] Processing CreateTablet for tablet 76f69ecff4db472ab0666dba1d88f481 (DEFAULT_TABLE table=test-table [id=aa4815c879d04cc1874bc3a5e18be2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:11.046312 20184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f69ecff4db472ab0666dba1d88f481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:11.046358 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.040159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.2.119.194:36873)
0619 08:18:11.040477 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.040481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.040510 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:18:11.040552 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.040554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.040571 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.040646 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:11.041039 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.042197 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.044612 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.045616 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.045681 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.045757 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.046220 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.046272 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":76,"threads_started":1}
I20260619 08:18:11.046430 20446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f69ecff4db472ab0666dba1d88f481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:11.046905 20315 tablet_service.cc:1467] Processing CreateTablet for tablet 76f69ecff4db472ab0666dba1d88f481 (DEFAULT_TABLE table=test-table [id=aa4815c879d04cc1874bc3a5e18be2a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:11.047215 20315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f69ecff4db472ab0666dba1d88f481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:11.049152 20526 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Bootstrap starting.
I20260619 08:18:11.049721 20527 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: Bootstrap starting.
I20260619 08:18:11.049934 20526 tablet_bootstrap.cc:654] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:11.050297 20526 log.cc:824] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:11.050410 20527 tablet_bootstrap.cc:654] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:11.050350 20528 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Bootstrap starting.
I20260619 08:18:11.050714 20527 log.cc:824] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:11.051172 20526 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: No bootstrap required, opened a new log
I20260619 08:18:11.051173 20528 tablet_bootstrap.cc:654] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:11.051262 20526 ts_tablet_manager.cc:1397] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:11.051494 20528 log.cc:824] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:11.051528 20527 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: No bootstrap required, opened a new log
I20260619 08:18:11.051582 20527 ts_tablet_manager.cc:1397] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:11.052321 20528 tablet_bootstrap.cc:492] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: No bootstrap required, opened a new log
I20260619 08:18:11.052387 20528 ts_tablet_manager.cc:1397] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:11.053279 20526 raft_consensus.cc:348] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.053498 20526 raft_consensus.cc:374] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:11.053539 20526 raft_consensus.cc:729] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6c15a27a5c48d59c8f7a82191c3277, State: Initialized, Role: FOLLOWER
I20260619 08:18:11.053666 20526 consensus_queue.cc:260] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.053920 20526 ts_tablet_manager.cc:1428] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 08:18:11.054044 20512 heartbeater.cc:503] Master 127.2.119.254:37017 was elected leader, sending a full tablet report...
I20260619 08:18:11.054162 20527 raft_consensus.cc:348] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.054318 20527 raft_consensus.cc:374] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:11.054358 20527 raft_consensus.cc:729] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fae5bc02c5c44d0b6a27612aaf27dc5, State: Initialized, Role: FOLLOWER
I20260619 08:18:11.054468 20527 consensus_queue.cc:260] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.054726 20527 ts_tablet_manager.cc:1428] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:11.054766 20250 heartbeater.cc:503] Master 127.2.119.254:37017 was elected leader, sending a full tablet report...
I20260619 08:18:11.055140 20528 raft_consensus.cc:348] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.055305 20528 raft_consensus.cc:374] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:11.055352 20528 raft_consensus.cc:729] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5d789caf61441b987ad92e6e011b79, State: Initialized, Role: FOLLOWER
I20260619 08:18:11.055469 20528 consensus_queue.cc:260] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.055727 20528 ts_tablet_manager.cc:1428] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:11.055902 20381 heartbeater.cc:503] Master 127.2.119.254:37017 was elected leader, sending a full tablet report...
W20260619 08:18:11.135936 20382 tablet.cc:2369] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:11.152009 20534 raft_consensus.cc:482] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:11.152175 20534 raft_consensus.cc:504] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.152566 20534 leader_election.cc:288] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fae5bc02c5c44d0b6a27612aaf27dc5 (127.2.119.193:41609), da6c15a27a5c48d59c8f7a82191c3277 (127.2.119.195:34525)
I20260619 08:18:11.156637 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.152874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:53247 (local address 127.2.119.193:41609)
0619 08:18:11.152917 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.152919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.153130 (+ 211us) server_negotiation.cc:407] Connection header received
0619 08:18:11.153182 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.153183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.153197 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.153251 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:11.153423 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.154156 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.155657 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.156270 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.156334 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.156342 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.156524 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.156563 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:11.156636 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.152894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:49925 (local address 127.2.119.195:34525)
0619 08:18:11.152940 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.152943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.152954 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:11.153065 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.153066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.153074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.153157 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:11.153382 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.154129 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.155658 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.156232 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.156325 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.156334 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.156524 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.156563 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:18:11.156637 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.152786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:34525 (local address 127.2.119.194:49925)
0619 08:18:11.152853 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.152867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.152939 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.153174 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.153175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.153177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:11.153275 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.153277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.154231 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.154232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.155561 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.155565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.156342 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.156343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.156423 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.156427 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.156492 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.156537 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":65}
I20260619 08:18:11.156644 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.152787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41609 (local address 127.2.119.194:53247)
0619 08:18:11.153054 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.153067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:11.153111 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:11.153271 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:11.153272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:11.153273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:11.153354 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.153356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.154231 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.154232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.155561 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:11.155565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.156345 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.156346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:11.156452 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.156456 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:11.156507 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 08:18:11.156537 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":129,"thread_start_us":103,"threads_started":1}
I20260619 08:18:11.157015 20204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f69ecff4db472ab0666dba1d88f481" candidate_uuid: "ab5d789caf61441b987ad92e6e011b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" is_pre_election: true
I20260619 08:18:11.157182 20204 raft_consensus.cc:2454] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab5d789caf61441b987ad92e6e011b79 in term 0.
I20260619 08:18:11.157241 20466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f69ecff4db472ab0666dba1d88f481" candidate_uuid: "ab5d789caf61441b987ad92e6e011b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6c15a27a5c48d59c8f7a82191c3277" is_pre_election: true
I20260619 08:18:11.157402 20466 raft_consensus.cc:2454] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab5d789caf61441b987ad92e6e011b79 in term 0.
I20260619 08:18:11.157416 20268 leader_election.cc:302] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fae5bc02c5c44d0b6a27612aaf27dc5, ab5d789caf61441b987ad92e6e011b79; no voters:
I20260619 08:18:11.157555 20534 raft_consensus.cc:2781] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:11.157611 20534 raft_consensus.cc:482] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:11.157635 20534 raft_consensus.cc:3037] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:11.158316 20534 raft_consensus.cc:504] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.158453 20534 leader_election.cc:288] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [CANDIDATE]: Term 1 election: Requested vote from peers 7fae5bc02c5c44d0b6a27612aaf27dc5 (127.2.119.193:41609), da6c15a27a5c48d59c8f7a82191c3277 (127.2.119.195:34525)
I20260619 08:18:11.158630 20204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f69ecff4db472ab0666dba1d88f481" candidate_uuid: "ab5d789caf61441b987ad92e6e011b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5"
I20260619 08:18:11.158638 20466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f69ecff4db472ab0666dba1d88f481" candidate_uuid: "ab5d789caf61441b987ad92e6e011b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da6c15a27a5c48d59c8f7a82191c3277"
I20260619 08:18:11.158702 20204 raft_consensus.cc:3037] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:11.158722 20466 raft_consensus.cc:3037] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:11.159420 20204 raft_consensus.cc:2454] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab5d789caf61441b987ad92e6e011b79 in term 1.
I20260619 08:18:11.159430 20466 raft_consensus.cc:2454] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab5d789caf61441b987ad92e6e011b79 in term 1.
I20260619 08:18:11.159576 20268 leader_election.cc:302] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fae5bc02c5c44d0b6a27612aaf27dc5, ab5d789caf61441b987ad92e6e011b79; no voters:
I20260619 08:18:11.159658 20534 raft_consensus.cc:2781] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:11.159790 20534 raft_consensus.cc:686] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [term 1 LEADER]: Becoming Leader. State: Replica: ab5d789caf61441b987ad92e6e011b79, State: Running, Role: LEADER
I20260619 08:18:11.159883 20534 consensus_queue.cc:237] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } }
I20260619 08:18:11.160665 20062 catalog_manager.cc:5495] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab5d789caf61441b987ad92e6e011b79 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "ab5d789caf61441b987ad92e6e011b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5d789caf61441b987ad92e6e011b79" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 } health_report { overall_health: UNKNOWN } } }
I20260619 08:18:11.188445 20522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.186122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.2.119.194:36873)
0619 08:18:11.186171 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:11.186173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:11.186183 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:11.186272 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:11.186273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:11.186290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:11.186342 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:11.186539 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.187002 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.187746 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:11.187930 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:11.187987 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:11.188119 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:11.188211 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:11.188271 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:11.188300 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 08:18:11.188316 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:18:11.190424 20466 raft_consensus.cc:1261] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 1 FOLLOWER]: Refusing update from remote peer ab5d789caf61441b987ad92e6e011b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:11.190442 20204 raft_consensus.cc:1261] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 1 FOLLOWER]: Refusing update from remote peer ab5d789caf61441b987ad92e6e011b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:11.190825 20534 consensus_queue.cc:1035] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da6c15a27a5c48d59c8f7a82191c3277" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:11.191001 20536 consensus_queue.cc:1035] T 76f69ecff4db472ab0666dba1d88f481 P ab5d789caf61441b987ad92e6e011b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fae5bc02c5c44d0b6a27612aaf27dc5" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 41609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:11.193286 20539 mvcc.cc:205] Tried to move back new op lower bound from 7298486645513003008 to 7298486645391441920. Current Snapshot: MvccSnapshot[applied={T|T < 7298486645513003008}]
I20260619 08:18:11.193552 20541 mvcc.cc:205] Tried to move back new op lower bound from 7298486645513003008 to 7298486645391441920. Current Snapshot: MvccSnapshot[applied={T|T < 7298486645513003008}]
I20260619 08:18:11.194048 20540 mvcc.cc:205] Tried to move back new op lower bound from 7298486645513003008 to 7298486645391441920. Current Snapshot: MvccSnapshot[applied={T|T < 7298486645513003008}]
I20260619 08:18:11.196281 20060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:11.196386 20060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:11.197158 20060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5091d4d7a63641b98c3b901a89fb973b: Sending DeleteTablet for 3 replicas of tablet 76f69ecff4db472ab0666dba1d88f481
I20260619 08:18:11.197486 20315 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f69ecff4db472ab0666dba1d88f481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:11 UTC) from {username='slave'} at 127.0.0.1:44594
I20260619 08:18:11.197503 20184 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f69ecff4db472ab0666dba1d88f481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:11 UTC) from {username='slave'} at 127.0.0.1:53412
I20260619 08:18:11.197530 20446 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f69ecff4db472ab0666dba1d88f481 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:11 UTC) from {username='slave'} at 127.0.0.1:52794
I20260619 08:18:11.197778 20549 tablet_replica.cc:330] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5: stopping tablet replica
I20260619 08:18:11.197871 20549 raft_consensus.cc:2229] T 76f69ecff4db472ab0666dba1d88f481 P 7fae5bc02c5c44d0b6a27612aaf27dc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:11.198005 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20122
I20260619 08:18:11.199594 20548 tablet_replica.cc:330] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: stopping tablet replica
I20260619 08:18:11.199700 20548 raft_consensus.cc:2229] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:11.199800 20548 raft_consensus.cc:2258] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:11.200243 20548 ts_tablet_manager.cc:1905] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:11.202224 20548 ts_tablet_manager.cc:1918] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:11.202301 20548 log.cc:1196] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/76f69ecff4db472ab0666dba1d88f481
I20260619 08:18:11.202550 20548 ts_tablet_manager.cc:1939] T 76f69ecff4db472ab0666dba1d88f481 P da6c15a27a5c48d59c8f7a82191c3277: Deleting consensus metadata
I20260619 08:18:11.202917 20049 catalog_manager.cc:4841] TS da6c15a27a5c48d59c8f7a82191c3277 (127.2.119.195:34525): tablet 76f69ecff4db472ab0666dba1d88f481 (table test-table [id=aa4815c879d04cc1874bc3a5e18be2a0]) successfully deleted
W20260619 08:18:11.203523 20046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:41609 (error 108)
I20260619 08:18:11.204777 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20253
I20260619 08:18:11.205765 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.205622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:41609 (local address 127.0.0.1:53414)
0619 08:18:11.205683 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.205729 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:41609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 08:18:11.205965 20046 catalog_manager.cc:4584] TS 7fae5bc02c5c44d0b6a27612aaf27dc5 (127.2.119.193:41609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76f69ecff4db472ab0666dba1d88f481: Network error: Client connection negotiation failed: client connection to 127.2.119.193:41609: connect: Connection refused (error 111)
I20260619 08:18:11.210556 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20384
I20260619 08:18:11.210579 20521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:11.210444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:36873 (local address 127.0.0.1:44610)
0619 08:18:11.210512 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:11.210543 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.194:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 08:18:11.216004 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20031
2026-06-19T08:18:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (767 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:11 dist-test-slave-d0b5 krb5kdc[20554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:11 dist-test-slave-d0b5 krb5kdc[20554](info): set up 2 sockets
Jun 19 08:18:11 dist-test-slave-d0b5 krb5kdc[20554](info): commencing operation
krb5kdc: starting...
W20260619 08:18:12.258126 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:12.375618 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39803
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:39803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:12.455919 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:12.456115 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:12.456137 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:12.457288 20567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:12.457352 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:12.457366 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:12.457379 20567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:12.457391 20567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:12.458860 20567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:39803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:12.459110 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:12.459358 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:12.464586 20572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.464637 20567 server_base.cc:1034] running on GCE node
W20260619 08:18:12.464600 20575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.464600 20573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.465051 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:12.465386 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:12.466575 20567 hybrid_clock.cc:648] HybridClock initialized: now 1781857092466564 us; error 28 us; skew 500 ppm
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:12.469144 20567 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:12.469586 20567 webserver.cc:460] Webserver started at http://127.2.119.254:34101/ using document root <none> and password file <none>
I20260619 08:18:12.469825 20567 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:12.469870 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:12.469959 20567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:12.470929 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "27534b5dde81427a80f66ba1d24e9733"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "7032c517c7cae6107d15a8a34c3317e4"
server_key_iv: "b4b87a6833743baa92f12185e30cdba6"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.471218 20567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "27534b5dde81427a80f66ba1d24e9733"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "7032c517c7cae6107d15a8a34c3317e4"
server_key_iv: "b4b87a6833743baa92f12185e30cdba6"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.472470 20567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:12.473243 20582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.473503 20567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:12.473596 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "27534b5dde81427a80f66ba1d24e9733"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "7032c517c7cae6107d15a8a34c3317e4"
server_key_iv: "b4b87a6833743baa92f12185e30cdba6"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.473652 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:12.484450 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:12.486423 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:12.486527 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:12.490258 20567 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:39803
I20260619 08:18:12.490288 20634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:39803 every 8 connection(s)
I20260619 08:18:12.490626 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:12.491478 20635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:12.493732 20635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: Bootstrap starting.
I20260619 08:18:12.494284 20635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:12.494594 20635 log.cc:824] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:12.495332 20635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: No bootstrap required, opened a new log
I20260619 08:18:12.496924 20635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } }
I20260619 08:18:12.497080 20635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:12.497126 20635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27534b5dde81427a80f66ba1d24e9733, State: Initialized, Role: FOLLOWER
I20260619 08:18:12.497332 20635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } }
I20260619 08:18:12.497434 20635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:12.497473 20635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:12.497525 20635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:12.498034 20635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } }
I20260619 08:18:12.498152 20635 leader_election.cc:302] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27534b5dde81427a80f66ba1d24e9733; no voters:
I20260619 08:18:12.498332 20635 leader_election.cc:288] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:12.498356 20638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:12.498584 20638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [term 1 LEADER]: Becoming Leader. State: Replica: 27534b5dde81427a80f66ba1d24e9733, State: Running, Role: LEADER
I20260619 08:18:12.498683 20635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:12.498708 20638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } }
I20260619 08:18:12.499147 20639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27534b5dde81427a80f66ba1d24e9733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } } }
I20260619 08:18:12.499231 20639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:12.499383 20640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27534b5dde81427a80f66ba1d24e9733. Latest consensus state: current_term: 1 leader_uuid: "27534b5dde81427a80f66ba1d24e9733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27534b5dde81427a80f66ba1d24e9733" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39803 } } }
I20260619 08:18:12.499470 20640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:12.500197 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20567
I20260619 08:18:12.500304 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:12.500564 2527 external_mini_cluster.cc:1428] Setting key 5a18ef3dede0cc3a573f828966193dce
W20260619 08:18:12.500560 20654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:12.500603 20654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:12.500665 20647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:12.500842 20647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:12.502349 20647 catalog_manager.cc:1347] Generated new cluster ID: c57555dab7214de2b401d8736bb520f1
I20260619 08:18:12.502399 20647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:12.512279 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:12.501042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44958 (local address 127.2.119.254:39803)
0619 08:18:12.501224 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:12.501234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:12.501269 (+ 35us) server_negotiation.cc:407] Connection header received
0619 08:18:12.501412 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:12.501417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:12.501497 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:12.501635 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:12.502450 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.503836 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.504945 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.505565 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.505662 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:12.508437 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:12.508443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:12.508444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:12.508461 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:12.510628 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.511014 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.511018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.511018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.511146 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.511366 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.511367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.511368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.511675 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:12.511831 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:12.511922 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:18:12.512063 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260619 08:18:12.522018 20647 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:12.522720 20647 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:12.538551 20647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: Generated new TSK 0
I20260619 08:18:12.538753 20647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:18:12.568260 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:12.652760 20661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:12.652968 20661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:12.652992 20661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:12.654225 20661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:12.654268 20661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:12.654296 20661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:12.655714 20661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:12.655973 20661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:12.656214 20661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:12.661290 20667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.661341 20666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.661401 20669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.661571 20661 server_base.cc:1034] running on GCE node
I20260619 08:18:12.661779 20661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:12.661978 20661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:12.663143 20661 hybrid_clock.cc:648] HybridClock initialized: now 1781857092663130 us; error 25 us; skew 500 ppm
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:12.665769 20661 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:18:12.666214 20661 webserver.cc:460] Webserver started at http://127.2.119.193:37813/ using document root <none> and password file <none>
I20260619 08:18:12.666450 20661 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:12.666517 20661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:12.666668 20661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:12.667891 20661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "95b2bf289e964945b05931e414cbb3a2"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "446f4bff3b4b2c549feb5e60608804e2"
server_key_iv: "f65891cf37b108c7297b55d104999ed8"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.668262 20661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "95b2bf289e964945b05931e414cbb3a2"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "446f4bff3b4b2c549feb5e60608804e2"
server_key_iv: "f65891cf37b108c7297b55d104999ed8"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.669675 20661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:12.670452 20676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.670645 20661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:18:12.670749 20661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "95b2bf289e964945b05931e414cbb3a2"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "446f4bff3b4b2c549feb5e60608804e2"
server_key_iv: "f65891cf37b108c7297b55d104999ed8"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.670825 20661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:12.682170 20661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:12.684005 20661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:12.684161 20661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:12.684381 20661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:12.684681 20661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:12.684736 20661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.684777 20661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:12.684805 20661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.690526 20661 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:43687
I20260619 08:18:12.690579 20789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:43687 every 8 connection(s)
I20260619 08:18:12.690871 20661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:18:12.693003 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20661
I20260619 08:18:12.693153 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:12.693442 2527 external_mini_cluster.cc:1428] Setting key 6e4561d51161067eb5c1744a4aa22ec8
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:12.701725 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:12.691545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:48933 (local address 127.2.119.254:39803)
0619 08:18:12.691620 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:12.691622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:12.691794 (+ 172us) server_negotiation.cc:407] Connection header received
0619 08:18:12.692247 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:12.692248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:12.692261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:12.692312 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:12.693252 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.694118 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.695135 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.695528 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.695614 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:12.698166 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:12.698167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:12.698168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:12.698176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:12.700135 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.700617 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.700619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.700619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.700683 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.700983 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.700984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.700984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.701130 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:12.701210 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:12.701532 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 08:18:12.701623 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 08:18:12.701814 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:12.691492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39803 (local address 127.2.119.193:48933)
0619 08:18:12.691681 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:12.691715 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:12.692169 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:12.692364 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:12.692367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:12.692917 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:12.693100 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:12.693105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.694216 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.694216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:12.695034 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:12.695036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.695748 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.695750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:12.695915 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:12.696441 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:12.696452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:12.698067 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:12.700216 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:12.700222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:12.700224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:12.700532 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:12.700772 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:12.700773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:12.700774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:12.700902 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:12.701205 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:12.701210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:12.701393 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 08:18:12.701485 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:18:12.701622 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260619 08:18:12.702358 20790 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39803
I20260619 08:18:12.702471 20790 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:12.702646 20790 heartbeater.cc:511] Master 127.2.119.254:39803 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
I20260619 08:18:12.703250 20599 ts_manager.cc:194] Registered new tserver with Master: 95b2bf289e964945b05931e414cbb3a2 (127.2.119.193:43687)
I20260619 08:18:12.703864 20599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:48933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:18:12.720749 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:12.806176 20795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:12.806375 20795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:12.806399 20795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:12.807478 20795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:12.807510 20795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:12.807535 20795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:12.808876 20795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:12.809049 20795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:12.809216 20795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:12.814220 20803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.814227 20801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.814508 20795 server_base.cc:1034] running on GCE node
W20260619 08:18:12.814231 20800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.814733 20795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:12.814975 20795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:12.816133 20795 hybrid_clock.cc:648] HybridClock initialized: now 1781857092816122 us; error 28 us; skew 500 ppm
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:12.818657 20795 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:18:12.819134 20795 webserver.cc:460] Webserver started at http://127.2.119.194:32839/ using document root <none> and password file <none>
I20260619 08:18:12.819367 20795 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:12.819434 20795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:12.819586 20795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:12.820638 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "e4f825e0a242461a9b35d498c0a346fc"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "d539839b721c29e20be7f8c752713f4f"
server_key_iv: "824d9493cce8c9b91272d91a59bba15a"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.821008 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "e4f825e0a242461a9b35d498c0a346fc"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "d539839b721c29e20be7f8c752713f4f"
server_key_iv: "824d9493cce8c9b91272d91a59bba15a"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.822418 20795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:12.823215 20810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.823406 20795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:12.823505 20795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "e4f825e0a242461a9b35d498c0a346fc"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "d539839b721c29e20be7f8c752713f4f"
server_key_iv: "824d9493cce8c9b91272d91a59bba15a"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.823590 20795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:12.848925 20795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:12.851543 20795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:12.851723 20795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:12.851998 20795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:12.852334 20795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:12.852389 20795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.852432 20795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:12.852460 20795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.858914 20795 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:35109
I20260619 08:18:12.858978 20923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:35109 every 8 connection(s)
I20260619 08:18:12.859288 20795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:12.865803 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20795
I20260619 08:18:12.865918 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:12.866158 2527 external_mini_cluster.cc:1428] Setting key ff13a9b1583603c821cdd2ed785b1565
I20260619 08:18:12.869239 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:12.859982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:40125 (local address 127.2.119.254:39803)
0619 08:18:12.860059 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:12.860061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:12.860355 (+ 294us) server_negotiation.cc:407] Connection header received
0619 08:18:12.860872 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:12.860873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:12.860888 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:12.860971 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:12.862005 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.862486 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.863443 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.863623 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.863676 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:12.865840 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:12.865841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:12.865842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:12.865851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:12.867683 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.868182 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.868183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.868183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.868237 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:12.868541 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:12.868542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:12.868542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:12.868696 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:12.868773 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:12.869044 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 08:18:12.869138 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 08:18:12.869359 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:12.860007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39803 (local address 127.2.119.194:40125)
0619 08:18:12.860224 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:12.860263 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:12.860726 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:12.861042 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:12.861046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:12.861700 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:12.861896 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:12.861900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.862589 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.862590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:12.863364 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:12.863366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:12.863698 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:12.863699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:12.863798 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:12.864279 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:12.864288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:12.865744 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:12.867761 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:12.867768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:12.867771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:12.868099 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:12.868300 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:12.868301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:12.868301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:12.868458 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:12.868770 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:12.868777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:12.868907 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 08:18:12.868997 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:18:12.869122 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260619 08:18:12.869858 20924 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39803
I20260619 08:18:12.869940 20924 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:12.870136 20924 heartbeater.cc:511] Master 127.2.119.254:39803 requested a full tablet report, sending...
I20260619 08:18:12.870527 20599 ts_manager.cc:194] Registered new tserver with Master: e4f825e0a242461a9b35d498c0a346fc (127.2.119.194:35109)
I20260619 08:18:12.871052 20599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:40125
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:18:12.892072 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:12.978159 20929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:12.978343 20929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:12.978366 20929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:12.979419 20929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:12.979453 20929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:12.979478 20929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:12.980824 20929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:39803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:12.981005 20929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:12.981166 20929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:12.986356 20935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.986392 20937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:12.986370 20934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:12.986397 20929 server_base.cc:1034] running on GCE node
I20260619 08:18:12.986822 20929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:12.987099 20929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:12.988271 20929 hybrid_clock.cc:648] HybridClock initialized: now 1781857092988229 us; error 73 us; skew 500 ppm
Jun 19 08:18:12 dist-test-slave-d0b5 krb5kdc[20554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:12.990712 20929 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:18:12.991151 20929 webserver.cc:460] Webserver started at http://127.2.119.195:42131/ using document root <none> and password file <none>
I20260619 08:18:12.991390 20929 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:12.991461 20929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:12.991611 20929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:12.992684 20929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "afa963cce1b94aa48839d57b46b0fa3b"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "0ef6ebd7aad685a7fa010a2162645b44"
server_key_iv: "146e15741863420a558a4c96f616198c"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.993048 20929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "afa963cce1b94aa48839d57b46b0fa3b"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "0ef6ebd7aad685a7fa010a2162645b44"
server_key_iv: "146e15741863420a558a4c96f616198c"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.994484 20929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:12.995265 20944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:12.995472 20929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:12.995579 20929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "afa963cce1b94aa48839d57b46b0fa3b"
format_stamp: "Formatted at 2026-06-19 08:18:12 on dist-test-slave-d0b5"
server_key: "0ef6ebd7aad685a7fa010a2162645b44"
server_key_iv: "146e15741863420a558a4c96f616198c"
server_key_version: "encryptionkey@0"
I20260619 08:18:12.995666 20929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:13.037184 20929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:13.039698 20929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:13.039876 20929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:13.040122 20929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:13.040433 20929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:13.040488 20929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.040530 20929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:13.040558 20929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.046598 20929 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:45947
I20260619 08:18:13.046797 21057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:45947 every 8 connection(s)
I20260619 08:18:13.047151 20929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:18:13.047997 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 20929
I20260619 08:18:13.048107 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:13.048293 2527 external_mini_cluster.cc:1428] Setting key 24dcc1fd80fcaf8dd02b200b484e716e
Jun 19 08:18:13 dist-test-slave-d0b5 krb5kdc[20554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857092, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:13.056571 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.048189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:60119 (local address 127.2.119.254:39803)
0619 08:18:13.048236 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.048238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.048271 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:18:13.048909 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.048910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.048922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.048969 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:13.050053 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.050556 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.051553 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.051745 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.051830 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.053956 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:13.053957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:13.053958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:13.053965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:13.055429 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:13.055810 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:13.055811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:13.055811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:13.055861 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:13.056065 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:13.056066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:13.056066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:13.056161 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:13.056209 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.056444 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.056507 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 08:18:13.056780 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.047948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:39803 (local address 127.2.119.195:60119)
0619 08:18:13.048133 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.048183 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.048810 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.049037 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.049041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.049711 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:13.049912 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.049917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.050660 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.050661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.051467 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.051468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.051839 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.051840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.051977 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.052441 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:13.052451 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:13.053873 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:13.055500 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:13.055505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:13.055507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:13.055744 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:13.055919 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:13.055919 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:13.055920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:13.056023 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:13.056216 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:13.056220 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:13.056331 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.056412 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.056535 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260619 08:18:13.057353 21058 heartbeater.cc:347] Connected to a master server at 127.2.119.254:39803
I20260619 08:18:13.057436 21058 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:13.057580 21058 heartbeater.cc:511] Master 127.2.119.254:39803 requested a full tablet report, sending...
I20260619 08:18:13.057948 20599 ts_manager.cc:194] Registered new tserver with Master: afa963cce1b94aa48839d57b46b0fa3b (127.2.119.195:45947)
I20260619 08:18:13.058326 20599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:60119
I20260619 08:18:13.060050 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:13.067711 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.061383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.2.119.254:39803)
0619 08:18:13.061429 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.061431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.061574 (+ 143us) server_negotiation.cc:407] Connection header received
0619 08:18:13.061610 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.061611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.061619 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.061669 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:13.062249 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.062711 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.063473 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.063656 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.063720 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.065000 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:13.065002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:13.065002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:13.065008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:13.066486 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:13.066930 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:13.066931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:13.066931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:13.066983 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:13.067192 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:13.067192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:13.067193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:13.067315 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:13.067394 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.067573 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.067620 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:18:13.069379 20599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:13.070940 20599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:13.077751 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53292 (local address 127.2.119.194:35109)
0619 08:18:13.073946 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.073950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.073957 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:13.074051 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.074053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.074069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.074297 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:13.074374 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075109 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.076635 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077231 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.077404 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.077474 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.077622 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.077667 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260619 08:18:13.077739 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35109 (local address 127.0.0.1:53292)
0619 08:18:13.073628 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.073639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.073692 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.074168 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.074169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.074172 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:13.074222 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.074224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075207 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.075208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.076542 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.076545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077410 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.077411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.077490 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.077503 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.077604 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.077662 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":93,"threads_started":1}
I20260619 08:18:13.077900 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43687 (local address 127.0.0.1:44342)
0619 08:18:13.073492 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.073529 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.073595 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.074187 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.074188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.074190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:13.074224 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.074225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075638 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.075639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.076616 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.076618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077628 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.077629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.077699 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.077707 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.077738 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.077783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":12,"thread_start_us":65,"threads_started":1}
I20260619 08:18:13.078121 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:45947 (local address 127.0.0.1:58158)
0619 08:18:13.073767 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.073775 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.073807 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.074128 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.074132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.074134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:13.074207 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.074209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075603 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.075605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.076755 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.076757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077626 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.077628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.077699 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.077705 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.077738 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.077783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":39,"thread_start_us":94,"threads_started":1}
I20260619 08:18:13.078438 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.2.119.195:45947)
0619 08:18:13.073793 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.073799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.073810 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:13.073917 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.073919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.073937 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.074289 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:13.074362 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075455 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.076827 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077522 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.078222 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.078280 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.078311 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.078358 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260619 08:18:13.078614 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.073573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.2.119.193:43687)
0619 08:18:13.074025 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.074029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.074039 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:13.074077 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.074079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.074094 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.074296 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:13.074379 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.075509 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.076732 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.077522 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.078108 (+ 586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.078370 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.078393 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.078433 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":89,"threads_started":1}
I20260619 08:18:13.078931 20858 tablet_service.cc:1467] Processing CreateTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 (DEFAULT_TABLE table=test-table [id=84638fab68f34a728338dd6503416bc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:13.079350 20858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3187cc08a6a4426994b9507d26ed9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:13.079283 20724 tablet_service.cc:1467] Processing CreateTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 (DEFAULT_TABLE table=test-table [id=84638fab68f34a728338dd6503416bc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:13.079298 20992 tablet_service.cc:1467] Processing CreateTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 (DEFAULT_TABLE table=test-table [id=84638fab68f34a728338dd6503416bc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:13.079574 20724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3187cc08a6a4426994b9507d26ed9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:13.079594 20992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3187cc08a6a4426994b9507d26ed9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:13.082185 21075 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Bootstrap starting.
I20260619 08:18:13.082785 21076 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Bootstrap starting.
I20260619 08:18:13.082898 21075 tablet_bootstrap.cc:654] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:13.083508 21075 log.cc:824] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:13.083559 21076 tablet_bootstrap.cc:654] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:13.084100 21076 log.cc:824] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:13.084332 21075 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: No bootstrap required, opened a new log
I20260619 08:18:13.084405 21075 ts_tablet_manager.cc:1397] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:13.084455 21077 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Bootstrap starting.
I20260619 08:18:13.084865 21076 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: No bootstrap required, opened a new log
I20260619 08:18:13.084952 21076 ts_tablet_manager.cc:1397] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:13.085075 21077 tablet_bootstrap.cc:654] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:13.085631 21077 log.cc:824] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:13.086402 21077 tablet_bootstrap.cc:492] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: No bootstrap required, opened a new log
I20260619 08:18:13.086468 21077 ts_tablet_manager.cc:1397] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:13.086665 21075 raft_consensus.cc:348] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.086849 21075 raft_consensus.cc:374] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:13.086889 21075 raft_consensus.cc:729] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b2bf289e964945b05931e414cbb3a2, State: Initialized, Role: FOLLOWER
I20260619 08:18:13.086838 21076 raft_consensus.cc:348] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.087035 21076 raft_consensus.cc:374] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:13.087028 21075 consensus_queue.cc:260] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.087075 21076 raft_consensus.cc:729] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa963cce1b94aa48839d57b46b0fa3b, State: Initialized, Role: FOLLOWER
I20260619 08:18:13.087209 21076 consensus_queue.cc:260] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.087306 21075 ts_tablet_manager.cc:1428] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:18:13.087383 20790 heartbeater.cc:503] Master 127.2.119.254:39803 was elected leader, sending a full tablet report...
I20260619 08:18:13.087482 21076 ts_tablet_manager.cc:1428] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:18:13.087589 21058 heartbeater.cc:503] Master 127.2.119.254:39803 was elected leader, sending a full tablet report...
I20260619 08:18:13.088414 21077 raft_consensus.cc:348] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.088593 21077 raft_consensus.cc:374] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:13.088630 21077 raft_consensus.cc:729] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f825e0a242461a9b35d498c0a346fc, State: Initialized, Role: FOLLOWER
I20260619 08:18:13.088735 21077 consensus_queue.cc:260] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.088992 21077 ts_tablet_manager.cc:1428] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 08:18:13.089030 20924 heartbeater.cc:503] Master 127.2.119.254:39803 was elected leader, sending a full tablet report...
W20260619 08:18:13.109828 20925 tablet.cc:2369] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:13.191624 20791 tablet.cc:2369] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:13.297748 21059 tablet.cc:2369] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:13.387265 21082 raft_consensus.cc:482] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:13.387411 21082 raft_consensus.cc:504] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.387780 21082 leader_election.cc:288] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95b2bf289e964945b05931e414cbb3a2 (127.2.119.193:43687), e4f825e0a242461a9b35d498c0a346fc (127.2.119.194:35109)
I20260619 08:18:13.390791 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.387943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43687 (local address 127.2.119.195:40939)
0619 08:18:13.388013 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.388026 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.388077 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.388264 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.388266 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.388268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:13.388345 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.388348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.389079 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.389080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.389931 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.389933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.390523 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.390609 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.390614 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.390672 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.390701 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260619 08:18:13.390806 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.388056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:40939 (local address 127.2.119.193:43687)
0619 08:18:13.388099 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.388101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.388131 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:18:13.388162 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.388164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.388176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.388235 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:13.388426 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.388986 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390039 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.390441 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390500 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.390506 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.390686 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.390731 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:13.390908 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.387981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:35109 (local address 127.2.119.195:50085)
0619 08:18:13.388159 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.388169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.388228 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.388397 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.388398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.388399 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:13.388470 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.388472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.389182 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.389183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.390064 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.390066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.390616 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.390703 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.390707 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.390763 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.390798 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":5,"thread_start_us":128,"threads_started":1}
I20260619 08:18:13.390902 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.388027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:50085 (local address 127.2.119.194:35109)
0619 08:18:13.388077 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.388080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.388247 (+ 167us) server_negotiation.cc:407] Connection header received
0619 08:18:13.388281 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.388282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.388292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.388353 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:13.388544 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.389062 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390150 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.390530 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.390601 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.390609 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.390782 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.390827 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:18:13.391149 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3187cc08a6a4426994b9507d26ed9a2" candidate_uuid: "afa963cce1b94aa48839d57b46b0fa3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b2bf289e964945b05931e414cbb3a2" is_pre_election: true
I20260619 08:18:13.391347 20744 raft_consensus.cc:2454] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa963cce1b94aa48839d57b46b0fa3b in term 0.
I20260619 08:18:13.391325 20878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3187cc08a6a4426994b9507d26ed9a2" candidate_uuid: "afa963cce1b94aa48839d57b46b0fa3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f825e0a242461a9b35d498c0a346fc" is_pre_election: true
I20260619 08:18:13.391487 20878 raft_consensus.cc:2454] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afa963cce1b94aa48839d57b46b0fa3b in term 0.
I20260619 08:18:13.391620 20947 leader_election.cc:302] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95b2bf289e964945b05931e414cbb3a2, afa963cce1b94aa48839d57b46b0fa3b; no voters:
I20260619 08:18:13.391830 21082 raft_consensus.cc:2781] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:13.391897 21082 raft_consensus.cc:482] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:13.391922 21082 raft_consensus.cc:3037] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:13.392652 21082 raft_consensus.cc:504] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.392786 21082 leader_election.cc:288] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [CANDIDATE]: Term 1 election: Requested vote from peers 95b2bf289e964945b05931e414cbb3a2 (127.2.119.193:43687), e4f825e0a242461a9b35d498c0a346fc (127.2.119.194:35109)
I20260619 08:18:13.392939 20878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3187cc08a6a4426994b9507d26ed9a2" candidate_uuid: "afa963cce1b94aa48839d57b46b0fa3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f825e0a242461a9b35d498c0a346fc"
I20260619 08:18:13.392962 20744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3187cc08a6a4426994b9507d26ed9a2" candidate_uuid: "afa963cce1b94aa48839d57b46b0fa3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b2bf289e964945b05931e414cbb3a2"
I20260619 08:18:13.393014 20878 raft_consensus.cc:3037] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:13.393059 20744 raft_consensus.cc:3037] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:13.393719 20878 raft_consensus.cc:2454] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa963cce1b94aa48839d57b46b0fa3b in term 1.
I20260619 08:18:13.393893 20945 leader_election.cc:302] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afa963cce1b94aa48839d57b46b0fa3b, e4f825e0a242461a9b35d498c0a346fc; no voters:
I20260619 08:18:13.393915 20744 raft_consensus.cc:2454] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afa963cce1b94aa48839d57b46b0fa3b in term 1.
I20260619 08:18:13.394008 21082 raft_consensus.cc:2781] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:13.394176 21082 raft_consensus.cc:686] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 1 LEADER]: Becoming Leader. State: Replica: afa963cce1b94aa48839d57b46b0fa3b, State: Running, Role: LEADER
I20260619 08:18:13.394280 21082 consensus_queue.cc:237] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } }
I20260619 08:18:13.395084 20598 catalog_manager.cc:5495] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b reported cstate change: term changed from 0 to 1, leader changed from <none> to afa963cce1b94aa48839d57b46b0fa3b (127.2.119.195). New cstate: current_term: 1 leader_uuid: "afa963cce1b94aa48839d57b46b0fa3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afa963cce1b94aa48839d57b46b0fa3b" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 45947 } health_report { overall_health: HEALTHY } } }
I20260619 08:18:13.432049 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.429618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.2.119.195:45947)
0619 08:18:13.429684 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.429686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.429698 (+ 12us) server_negotiation.cc:407] Connection header received
0619 08:18:13.429745 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.429753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.429771 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.429849 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:13.430029 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.430502 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.431254 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.431450 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.431523 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.431674 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:13.431771 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:13.431878 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.431938 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.431949 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 08:18:13.434165 20878 raft_consensus.cc:1261] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 1 FOLLOWER]: Refusing update from remote peer afa963cce1b94aa48839d57b46b0fa3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:13.434165 20744 raft_consensus.cc:1261] T f3187cc08a6a4426994b9507d26ed9a2 P 95b2bf289e964945b05931e414cbb3a2 [term 1 FOLLOWER]: Refusing update from remote peer afa963cce1b94aa48839d57b46b0fa3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:13.434540 21085 consensus_queue.cc:1035] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b2bf289e964945b05931e414cbb3a2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 43687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:13.434664 21082 consensus_queue.cc:1035] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f825e0a242461a9b35d498c0a346fc" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 35109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:13.437458 21088 mvcc.cc:205] Tried to move back new op lower bound from 7298486654703411200 to 7298486654543491072. Current Snapshot: MvccSnapshot[applied={T|T < 7298486654703411200}]
I20260619 08:18:13.437602 21090 mvcc.cc:205] Tried to move back new op lower bound from 7298486654703411200 to 7298486654543491072. Current Snapshot: MvccSnapshot[applied={T|T < 7298486654703411200}]
I20260619 08:18:13.437242 21089 mvcc.cc:205] Tried to move back new op lower bound from 7298486654703411200 to 7298486654543491072. Current Snapshot: MvccSnapshot[applied={T|T < 7298486654703411200}]
I20260619 08:18:13.440075 20598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:13.440176 20598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:13.440925 20598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27534b5dde81427a80f66ba1d24e9733: Sending DeleteTablet for 3 replicas of tablet f3187cc08a6a4426994b9507d26ed9a2
I20260619 08:18:13.441250 20858 tablet_service.cc:1514] Processing DeleteTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:13 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:53292
I20260619 08:18:13.441254 20724 tablet_service.cc:1514] Processing DeleteTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:13 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:44342
I20260619 08:18:13.441551 20992 tablet_service.cc:1514] Processing DeleteTablet for tablet f3187cc08a6a4426994b9507d26ed9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:13 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:58158
I20260619 08:18:13.441664 21098 tablet_replica.cc:330] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: stopping tablet replica
I20260619 08:18:13.441768 21098 raft_consensus.cc:2229] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:13.441789 21099 tablet_replica.cc:330] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: stopping tablet replica
I20260619 08:18:13.441937 21098 raft_consensus.cc:2258] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:13.441939 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20661
I20260619 08:18:13.441893 21099 raft_consensus.cc:2229] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:18:13.442696 21099 raft_consensus.cc:2258] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:13.443081 21099 ts_tablet_manager.cc:1905] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:13.443657 21098 ts_tablet_manager.cc:1905] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:13.444764 21099 ts_tablet_manager.cc:1918] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:13.444893 21099 log.cc:1196] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/f3187cc08a6a4426994b9507d26ed9a2
I20260619 08:18:13.445240 21099 ts_tablet_manager.cc:1939] T f3187cc08a6a4426994b9507d26ed9a2 P afa963cce1b94aa48839d57b46b0fa3b: Deleting consensus metadata
I20260619 08:18:13.445389 21098 ts_tablet_manager.cc:1918] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:13.445501 21098 log.cc:1196] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/f3187cc08a6a4426994b9507d26ed9a2
I20260619 08:18:13.445783 21098 ts_tablet_manager.cc:1939] T f3187cc08a6a4426994b9507d26ed9a2 P e4f825e0a242461a9b35d498c0a346fc: Deleting consensus metadata
I20260619 08:18:13.446143 20583 catalog_manager.cc:4841] TS e4f825e0a242461a9b35d498c0a346fc (127.2.119.194:35109): tablet f3187cc08a6a4426994b9507d26ed9a2 (table test-table [id=84638fab68f34a728338dd6503416bc3]) successfully deleted
I20260619 08:18:13.446151 20586 catalog_manager.cc:4841] TS afa963cce1b94aa48839d57b46b0fa3b (127.2.119.195:45947): tablet f3187cc08a6a4426994b9507d26ed9a2 (table test-table [id=84638fab68f34a728338dd6503416bc3]) successfully deleted
W20260619 08:18:13.450125 20585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:43687 (error 108)
I20260619 08:18:13.450348 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20795
I20260619 08:18:13.450939 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.450507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:43687 (local address 127.0.0.1:44346)
0619 08:18:13.450851 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.450892 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322}
W20260619 08:18:13.451071 20585 catalog_manager.cc:4584] TS 95b2bf289e964945b05931e414cbb3a2 (127.2.119.193:43687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f3187cc08a6a4426994b9507d26ed9a2: Network error: Client connection negotiation failed: client connection to 127.2.119.193:43687: connect: Connection refused (error 111)
I20260619 08:18:13.456025 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20929
I20260619 08:18:13.460572 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 20567
2026-06-19T08:18:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2244 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T08:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:13Z Disabled control of system clock
I20260619 08:18:13.487402 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45237
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:13.568403 21104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:13.568624 21104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:13.568643 21104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:13.569976 21104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:13.570020 21104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:13.570034 21104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:13.570046 21104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:13.570060 21104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:13.571506 21104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45237
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:13.571753 21104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:13.572060 21104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:13.576949 21112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:13.576946 21110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:13.577100 21104 server_base.cc:1034] running on GCE node
W20260619 08:18:13.577100 21109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:13.577551 21104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:13.577795 21104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:13.578979 21104 hybrid_clock.cc:648] HybridClock initialized: now 1781857093578955 us; error 34 us; skew 500 ppm
I20260619 08:18:13.584795 21104 webserver.cc:460] Webserver started at http://127.2.119.254:43453/ using document root <none> and password file <none>
I20260619 08:18:13.585078 21104 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:13.585158 21104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:13.585353 21104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:13.586531 21104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "e9ad4212f2784c34a5bfae8d2f4454b4"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "4f57a23edaad9ce02ca7a16d6e1b1e0e"
server_key_iv: "958d103e386349f6e2f3c2197fa8c2fd"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.586910 21104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "e9ad4212f2784c34a5bfae8d2f4454b4"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "4f57a23edaad9ce02ca7a16d6e1b1e0e"
server_key_iv: "958d103e386349f6e2f3c2197fa8c2fd"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.588197 21104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:13.589007 21118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.589208 21104 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:13.589337 21104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "e9ad4212f2784c34a5bfae8d2f4454b4"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "4f57a23edaad9ce02ca7a16d6e1b1e0e"
server_key_iv: "958d103e386349f6e2f3c2197fa8c2fd"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.589442 21104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:13.603297 21104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:13.603626 21104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:13.603793 21104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:13.607967 21104 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45237
I20260619 08:18:13.608045 21170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45237 every 8 connection(s)
I20260619 08:18:13.608402 21104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:13.609413 21171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:13.611687 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21104
I20260619 08:18:13.611702 21171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: Bootstrap starting.
I20260619 08:18:13.611802 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:13.611984 2527 external_mini_cluster.cc:1428] Setting key 657d8814f087b6ca068d8b4744313424
I20260619 08:18:13.612424 21171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:13.612847 21171 log.cc:824] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:13.613656 21171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: No bootstrap required, opened a new log
I20260619 08:18:13.615690 21171 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } }
I20260619 08:18:13.615902 21171 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:13.615952 21171 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9ad4212f2784c34a5bfae8d2f4454b4, State: Initialized, Role: FOLLOWER
I20260619 08:18:13.616073 21171 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } }
I20260619 08:18:13.616168 21171 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:13.616210 21171 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:13.616257 21171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:13.616788 21171 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } }
I20260619 08:18:13.616909 21171 leader_election.cc:302] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9ad4212f2784c34a5bfae8d2f4454b4; no voters:
I20260619 08:18:13.617074 21171 leader_election.cc:288] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:13.617249 21177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:13.617357 21171 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:13.617422 21177 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [term 1 LEADER]: Becoming Leader. State: Replica: e9ad4212f2784c34a5bfae8d2f4454b4, State: Running, Role: LEADER
I20260619 08:18:13.617512 21177 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } }
I20260619 08:18:13.617848 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } } }
I20260619 08:18:13.617992 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:13.618155 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.612523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.2.119.254:45237)
0619 08:18:13.612736 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.612743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.612768 (+ 25us) server_negotiation.cc:407] Connection header received
0619 08:18:13.613204 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.613211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.613268 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.613445 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:13.613684 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.614595 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.615759 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.616262 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.616392 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.617092 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:13.617095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:13.617096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:13.617343 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:13.617385 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:13.617401 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:13.617741 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.617777 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.617867 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260619 08:18:13.618186 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9ad4212f2784c34a5bfae8d2f4454b4. Latest consensus state: current_term: 1 leader_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9ad4212f2784c34a5bfae8d2f4454b4" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45237 } } }
I20260619 08:18:13.618450 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:13.618301 21185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:13.618942 21185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:13.622380 21185 catalog_manager.cc:1347] Generated new cluster ID: 8407b535e24643728ff411651faae164
I20260619 08:18:13.622435 21185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:13.632288 21185 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:13.632820 21185 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:13.638000 21185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: Generated new TSK 0
I20260619 08:18:13.638168 21185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:13.646940 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--builtin_ntp_servers=127.2.119.212:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:13.729103 21195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:13.729379 21195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:13.729440 21195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:13.730607 21195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:13.730648 21195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:13.730674 21195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:13.732117 21195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:13.732355 21195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:13.732646 21195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:13.738879 21200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:13.738927 21201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:13.738953 21195 server_base.cc:1034] running on GCE node
W20260619 08:18:13.739154 21203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:13.739383 21195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:13.739670 21195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:13.740921 21195 hybrid_clock.cc:648] HybridClock initialized: now 1781857093740909 us; error 34 us; skew 500 ppm
I20260619 08:18:13.742280 21195 webserver.cc:460] Webserver started at http://127.2.119.193:38437/ using document root <none> and password file <none>
I20260619 08:18:13.742612 21195 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:13.742688 21195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:13.742851 21195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:13.743973 21195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "b52358f31eb74ea7a447edc12fd35add"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "e2f9e0afbf23fea684599ad0abe7cff0"
server_key_iv: "4a0cb49017e4d6ab28f71f6db006915c"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.744431 21195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "b52358f31eb74ea7a447edc12fd35add"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "e2f9e0afbf23fea684599ad0abe7cff0"
server_key_iv: "4a0cb49017e4d6ab28f71f6db006915c"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.746151 21195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 08:18:13.747166 21209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.747326 21195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:13.747421 21195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "b52358f31eb74ea7a447edc12fd35add"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "e2f9e0afbf23fea684599ad0abe7cff0"
server_key_iv: "4a0cb49017e4d6ab28f71f6db006915c"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.747503 21195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:13.771937 21195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:13.772300 21195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:13.772464 21195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:13.772704 21195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:13.773049 21195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:13.773105 21195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.773146 21195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:13.773175 21195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.778978 21195 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:42887
I20260619 08:18:13.779021 21322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:42887 every 8 connection(s)
I20260619 08:18:13.779345 21195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:18:13.782454 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21195
I20260619 08:18:13.782579 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:13.782783 2527 external_mini_cluster.cc:1428] Setting key c8d3ca859509d48cae73b0fa81cde5da
I20260619 08:18:13.783752 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--builtin_ntp_servers=127.2.119.212:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:18:13.785322 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.779990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:54075 (local address 127.2.119.254:45237)
0619 08:18:13.780058 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.780060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.780356 (+ 296us) server_negotiation.cc:407] Connection header received
0619 08:18:13.780823 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.780824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.780833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.780884 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:13.781683 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.782247 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.783542 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.783863 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.783950 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.784973 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:13.784974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:13.784975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:13.784980 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:13.785000 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:13.785005 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:13.785071 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.785180 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.785207 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:18:13.785547 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.780064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45237 (local address 127.2.119.193:54075)
0619 08:18:13.780237 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.780271 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.780744 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.780936 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.780939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.780948 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:13.781589 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.781595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.782380 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.782381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.783454 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.783457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.784016 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.784017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.784180 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.784787 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:13.784788 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:13.784827 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:13.784854 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:13.785081 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:13.785089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:13.785092 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.785171 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.785272 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260619 08:18:13.785956 21323 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45237
I20260619 08:18:13.786041 21323 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:13.786239 21323 heartbeater.cc:511] Master 127.2.119.254:45237 requested a full tablet report, sending...
I20260619 08:18:13.786722 21135 ts_manager.cc:194] Registered new tserver with Master: b52358f31eb74ea7a447edc12fd35add (127.2.119.193:42887)
I20260619 08:18:13.787355 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.193:54075
W20260619 08:18:13.864825 21326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:13.865051 21326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:13.865087 21326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:13.866354 21326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:13.866422 21326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:13.866463 21326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:13.867826 21326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:13.868099 21326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:13.868408 21326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:18:13.873822 21326 server_base.cc:1034] running on GCE node
W20260619 08:18:13.873788 21331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:13.873785 21334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:13.873782 21332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:13.874356 21326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:13.874565 21326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:13.875761 21326 hybrid_clock.cc:648] HybridClock initialized: now 1781857093875743 us; error 42 us; skew 500 ppm
I20260619 08:18:13.877249 21326 webserver.cc:460] Webserver started at http://127.2.119.194:39311/ using document root <none> and password file <none>
I20260619 08:18:13.877522 21326 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:13.877599 21326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:13.877753 21326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:13.878887 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "1e3a91440dee447fbe0370bc875ecd7a"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "fe9e8a30eb740ac389587a7de76252d3"
server_key_iv: "6a6f44b8beb28d25431bb182a8c285bc"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.879256 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "1e3a91440dee447fbe0370bc875ecd7a"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "fe9e8a30eb740ac389587a7de76252d3"
server_key_iv: "6a6f44b8beb28d25431bb182a8c285bc"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.880637 21326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:13.881497 21340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.881676 21326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:13.881762 21326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "1e3a91440dee447fbe0370bc875ecd7a"
format_stamp: "Formatted at 2026-06-19 08:18:13 on dist-test-slave-d0b5"
server_key: "fe9e8a30eb740ac389587a7de76252d3"
server_key_iv: "6a6f44b8beb28d25431bb182a8c285bc"
server_key_version: "encryptionkey@0"
I20260619 08:18:13.881840 21326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:13.901495 21326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:13.901791 21326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:13.901932 21326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:13.902158 21326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:13.902486 21326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:13.902539 21326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.902581 21326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:13.902607 21326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:13.908164 21326 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:44007
I20260619 08:18:13.908211 21453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:44007 every 8 connection(s)
I20260619 08:18:13.908520 21326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:18:13.908963 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21326
I20260619 08:18:13.909085 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:13.909329 2527 external_mini_cluster.cc:1428] Setting key d4b4a01ac15e20e9a3725057cd4878f9
I20260619 08:18:13.910434 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--builtin_ntp_servers=127.2.119.212:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:18:13.914439 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.909230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:59375 (local address 127.2.119.254:45237)
0619 08:18:13.909280 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:13.909282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:13.909633 (+ 351us) server_negotiation.cc:407] Connection header received
0619 08:18:13.910185 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:13.910186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:13.910199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:13.910262 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:13.910989 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.911791 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.912762 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.913072 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.913141 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.914104 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:13.914106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:13.914107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:13.914112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:13.914131 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:13.914135 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:13.914200 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:13.914286 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 08:18:13.914323 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:13.914528 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:13.909244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45237 (local address 127.2.119.194:59375)
0619 08:18:13.909476 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:13.909523 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:13.910074 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:13.910312 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:13.910315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:13.910324 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:13.910905 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.910910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.911890 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.911891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.912662 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:13.912664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:13.913156 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:13.913157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:13.913276 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:13.913909 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:13.913909 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:13.913971 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:13.914003 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:13.914206 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:13.914210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:13.914213 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:13.914266 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 08:18:13.914348 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260619 08:18:13.914994 21454 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45237
I20260619 08:18:13.915097 21454 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:13.915267 21454 heartbeater.cc:511] Master 127.2.119.254:45237 requested a full tablet report, sending...
I20260619 08:18:13.915634 21135 ts_manager.cc:194] Registered new tserver with Master: 1e3a91440dee447fbe0370bc875ecd7a (127.2.119.194:44007)
I20260619 08:18:13.916177 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.194:59375
W20260619 08:18:13.992002 21457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:13.992255 21457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:13.992278 21457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:13.993546 21457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:13.993585 21457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:13.993611 21457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:13.994915 21457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:45237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:13.995149 21457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:13.995427 21457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:18:14.000856 21457 server_base.cc:1034] running on GCE node
W20260619 08:18:14.000803 21462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:14.000805 21463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:14.000810 21465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:14.001220 21457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:14.001544 21457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:14.002700 21457 hybrid_clock.cc:648] HybridClock initialized: now 1781857094002694 us; error 39 us; skew 500 ppm
I20260619 08:18:14.004050 21457 webserver.cc:460] Webserver started at http://127.2.119.195:42793/ using document root <none> and password file <none>
I20260619 08:18:14.004324 21457 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:14.004400 21457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:14.004596 21457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:14.005749 21457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "fb7d8cd1362143b8b99c78a8afae9ec3"
format_stamp: "Formatted at 2026-06-19 08:18:14 on dist-test-slave-d0b5"
server_key: "58349594165936a240b81c333ae71dca"
server_key_iv: "575182a6d33095a1e420bf2ac047f8bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:14.006049 21457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "fb7d8cd1362143b8b99c78a8afae9ec3"
format_stamp: "Formatted at 2026-06-19 08:18:14 on dist-test-slave-d0b5"
server_key: "58349594165936a240b81c333ae71dca"
server_key_iv: "575182a6d33095a1e420bf2ac047f8bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:14.007287 21457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:14.007944 21471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:14.008117 21457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:14.008184 21457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "fb7d8cd1362143b8b99c78a8afae9ec3"
format_stamp: "Formatted at 2026-06-19 08:18:14 on dist-test-slave-d0b5"
server_key: "58349594165936a240b81c333ae71dca"
server_key_iv: "575182a6d33095a1e420bf2ac047f8bb"
server_key_version: "encryptionkey@0"
I20260619 08:18:14.008242 21457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:14.035418 21457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:14.035691 21457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:14.035867 21457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:14.036118 21457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:14.036465 21457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:14.036499 21457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:14.036553 21457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:14.036580 21457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:14.042307 21457 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:35183
I20260619 08:18:14.042353 21584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:35183 every 8 connection(s)
I20260619 08:18:14.042681 21457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:18:14.045982 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21457
I20260619 08:18:14.046100 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:14.046308 2527 external_mini_cluster.cc:1428] Setting key 721ebfbe3c731c886a92361910cd37e0
I20260619 08:18:14.048560 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.043453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:44959 (local address 127.2.119.254:45237)
0619 08:18:14.043543 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.043546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.043741 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:18:14.044208 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.044209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.044221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.044271 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:14.045164 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.045726 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.047132 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.047394 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.047463 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.048342 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:14.048343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:14.048343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:14.048347 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:14.048362 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:14.048366 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:14.048410 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.048478 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.048499 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:18:14.048774 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.043423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:45237 (local address 127.2.119.195:44959)
0619 08:18:14.043613 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.043648 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.044108 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.044334 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.044338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.044347 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:14.045055 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.045061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.045901 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.045902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.047051 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.047054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.047486 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.047487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.047608 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.048206 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:14.048207 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:14.048242 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:14.048264 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:14.048412 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:14.048414 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:14.048417 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.048464 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.048537 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260619 08:18:14.049261 21585 heartbeater.cc:347] Connected to a master server at 127.2.119.254:45237
I20260619 08:18:14.049435 21585 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:14.049657 21585 heartbeater.cc:511] Master 127.2.119.254:45237 requested a full tablet report, sending...
I20260619 08:18:14.050092 21135 ts_manager.cc:194] Registered new tserver with Master: fb7d8cd1362143b8b99c78a8afae9ec3 (127.2.119.195:35183)
I20260619 08:18:14.050500 21135 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.195:44959
I20260619 08:18:14.058099 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:14.062634 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.059501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.2.119.254:45237)
0619 08:18:14.059590 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.059593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.059703 (+ 110us) server_negotiation.cc:407] Connection header received
0619 08:18:14.059799 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.059800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.059812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.059868 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:14.060038 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.060516 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.061322 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.061525 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.061600 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.062300 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:14.062301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:14.062301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:14.062306 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:14.062321 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:14.062325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:14.062449 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.062538 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.062550 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:18:14.064538 21135 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:14.066185 21135 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:14.073033 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.068599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:42887 (local address 127.0.0.1:57238)
0619 08:18:14.068915 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.068925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.068971 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.069676 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.069677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.069679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:14.069709 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.069709 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.070881 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.070882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.071965 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.071967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072720 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.072812 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.072827 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.072898 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.072954 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":90,"threads_started":1}
I20260619 08:18:14.073246 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.068762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.2.119.194:44007)
0619 08:18:14.069050 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.069060 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.069067 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:14.069098 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.069099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.069111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.069199 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:14.069381 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.070546 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072036 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072745 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.072924 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.073112 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.073163 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":89,"threads_started":1}
I20260619 08:18:14.073405 21594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.068583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:44007 (local address 127.0.0.1:43832)
0619 08:18:14.068744 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.068786 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.068914 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.069201 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.069204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.069206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:14.069274 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.069275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.070633 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.070633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.071940 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.071943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072958 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.073008 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.073011 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.073182 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.073229 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260619 08:18:14.074198 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.068691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.2.119.193:42887)
0619 08:18:14.069518 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.069522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.069531 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:14.069567 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.069569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.069583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.069666 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:14.069763 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.070806 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072053 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072625 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072715 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.072809 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.074097 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.074134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":148,"threads_started":1}
I20260619 08:18:14.074106 21388 tablet_service.cc:1467] Processing CreateTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 (DEFAULT_TABLE table=test-table [id=5f566a980737481192cb60f1c5ad30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:14.074474 21388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb649a1365ae4c30bf31f4c68b5eee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:14.074800 21257 tablet_service.cc:1467] Processing CreateTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 (DEFAULT_TABLE table=test-table [id=5f566a980737481192cb60f1c5ad30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:14.075098 21257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb649a1365ae4c30bf31f4c68b5eee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:14.075291 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.068786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:35183 (local address 127.0.0.1:40866)
0619 08:18:14.069337 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.069347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.069397 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.069657 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.069658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.069660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:14.069704 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.069705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072890 (+ 3185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.072891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.074022 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.074025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.075049 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.075050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.075112 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.075116 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.075185 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.075226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":78,"threads_started":1}
I20260619 08:18:14.075769 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.069018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.2.119.195:35183)
0619 08:18:14.069176 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.069180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.069398 (+ 218us) server_negotiation.cc:407] Connection header received
0619 08:18:14.069473 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.069475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.069490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.069571 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:14.070880 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.072788 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.074099 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.074911 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.075518 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.075610 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.075636 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.075679 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260619 08:18:14.076503 21519 tablet_service.cc:1467] Processing CreateTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 (DEFAULT_TABLE table=test-table [id=5f566a980737481192cb60f1c5ad30e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:14.076794 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb649a1365ae4c30bf31f4c68b5eee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:14.078110 21599 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Bootstrap starting.
I20260619 08:18:14.078392 21600 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Bootstrap starting.
I20260619 08:18:14.078914 21599 tablet_bootstrap.cc:654] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:14.078991 21600 tablet_bootstrap.cc:654] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:14.079262 21599 log.cc:824] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:14.079303 21600 log.cc:824] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:14.080070 21600 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: No bootstrap required, opened a new log
I20260619 08:18:14.080113 21603 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Bootstrap starting.
I20260619 08:18:14.080138 21600 ts_tablet_manager.cc:1397] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:14.080338 21599 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: No bootstrap required, opened a new log
I20260619 08:18:14.080431 21599 ts_tablet_manager.cc:1397] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:14.080756 21603 tablet_bootstrap.cc:654] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:14.081111 21603 log.cc:824] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:14.081878 21603 tablet_bootstrap.cc:492] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: No bootstrap required, opened a new log
I20260619 08:18:14.081945 21603 ts_tablet_manager.cc:1397] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 08:18:14.082512 21599 raft_consensus.cc:348] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.082664 21599 raft_consensus.cc:374] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:14.082705 21599 raft_consensus.cc:729] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3a91440dee447fbe0370bc875ecd7a, State: Initialized, Role: FOLLOWER
I20260619 08:18:14.082818 21599 consensus_queue.cc:260] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.082846 21600 raft_consensus.cc:348] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.083001 21600 raft_consensus.cc:374] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:14.083043 21600 raft_consensus.cc:729] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b52358f31eb74ea7a447edc12fd35add, State: Initialized, Role: FOLLOWER
I20260619 08:18:14.083087 21599 ts_tablet_manager.cc:1428] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:14.083153 21600 consensus_queue.cc:260] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.083280 21454 heartbeater.cc:503] Master 127.2.119.254:45237 was elected leader, sending a full tablet report...
I20260619 08:18:14.083408 21600 ts_tablet_manager.cc:1428] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260619 08:18:14.083541 21323 heartbeater.cc:503] Master 127.2.119.254:45237 was elected leader, sending a full tablet report...
I20260619 08:18:14.084405 21603 raft_consensus.cc:348] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.084561 21603 raft_consensus.cc:374] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:14.084602 21603 raft_consensus.cc:729] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7d8cd1362143b8b99c78a8afae9ec3, State: Initialized, Role: FOLLOWER
I20260619 08:18:14.084708 21603 consensus_queue.cc:260] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.084954 21603 ts_tablet_manager.cc:1428] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 08:18:14.085011 21585 heartbeater.cc:503] Master 127.2.119.254:45237 was elected leader, sending a full tablet report...
W20260619 08:18:14.159127 21455 tablet.cc:2369] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:14.280021 21324 tablet.cc:2369] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:14.293182 21586 tablet.cc:2369] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:14.326243 21605 raft_consensus.cc:482] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:14.326406 21605 raft_consensus.cc:504] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.326784 21605 leader_election.cc:288] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b52358f31eb74ea7a447edc12fd35add (127.2.119.193:42887), fb7d8cd1362143b8b99c78a8afae9ec3 (127.2.119.195:35183)
I20260619 08:18:14.330685 21595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.327095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:54699 (local address 127.2.119.193:42887)
0619 08:18:14.327138 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.327140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.327183 (+ 43us) server_negotiation.cc:407] Connection header received
0619 08:18:14.327215 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.327216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.327229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.327288 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:14.327445 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.328229 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.329729 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.330311 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.330393 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.330401 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.330574 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.330614 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:14.330685 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.327076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:56457 (local address 127.2.119.195:35183)
0619 08:18:14.327339 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.327341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.327348 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:14.327380 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.327380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.327388 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.327459 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:14.327598 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.328306 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.329753 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.330313 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.330393 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.330401 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.330574 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.330614 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:18:14.330696 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.326997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:42887 (local address 127.2.119.194:54699)
0619 08:18:14.327057 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.327070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.327115 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.327301 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.327302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.327304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:14.327363 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.327365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.328315 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.328316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.329620 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.329623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.330414 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.330415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.330487 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.330490 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.330550 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.330592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":8}
I20260619 08:18:14.330701 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.327010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:35183 (local address 127.2.119.194:56457)
0619 08:18:14.327196 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.327207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:14.327271 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:14.327479 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:14.327480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:14.327481 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:14.327528 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.327530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.328386 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.328387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.329620 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:14.329623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.330408 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.330409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:14.330490 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.330492 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:18:14.330550 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 08:18:14.330591 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":43,"thread_start_us":96,"threads_started":1}
I20260619 08:18:14.331085 21277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb649a1365ae4c30bf31f4c68b5eee87" candidate_uuid: "1e3a91440dee447fbe0370bc875ecd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b52358f31eb74ea7a447edc12fd35add" is_pre_election: true
I20260619 08:18:14.331254 21277 raft_consensus.cc:2454] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e3a91440dee447fbe0370bc875ecd7a in term 0.
I20260619 08:18:14.331382 21539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb649a1365ae4c30bf31f4c68b5eee87" candidate_uuid: "1e3a91440dee447fbe0370bc875ecd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" is_pre_election: true
I20260619 08:18:14.331503 21343 leader_election.cc:302] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e3a91440dee447fbe0370bc875ecd7a, b52358f31eb74ea7a447edc12fd35add; no voters:
I20260619 08:18:14.331547 21539 raft_consensus.cc:2454] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e3a91440dee447fbe0370bc875ecd7a in term 0.
I20260619 08:18:14.331646 21605 raft_consensus.cc:2781] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:14.331717 21605 raft_consensus.cc:482] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:14.331744 21605 raft_consensus.cc:3037] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:14.332427 21605 raft_consensus.cc:504] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.332530 21605 leader_election.cc:288] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [CANDIDATE]: Term 1 election: Requested vote from peers b52358f31eb74ea7a447edc12fd35add (127.2.119.193:42887), fb7d8cd1362143b8b99c78a8afae9ec3 (127.2.119.195:35183)
I20260619 08:18:14.332697 21539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb649a1365ae4c30bf31f4c68b5eee87" candidate_uuid: "1e3a91440dee447fbe0370bc875ecd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3"
I20260619 08:18:14.332702 21277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb649a1365ae4c30bf31f4c68b5eee87" candidate_uuid: "1e3a91440dee447fbe0370bc875ecd7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b52358f31eb74ea7a447edc12fd35add"
I20260619 08:18:14.332773 21539 raft_consensus.cc:3037] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:14.332794 21277 raft_consensus.cc:3037] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:14.333563 21539 raft_consensus.cc:2454] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e3a91440dee447fbe0370bc875ecd7a in term 1.
I20260619 08:18:14.333580 21277 raft_consensus.cc:2454] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e3a91440dee447fbe0370bc875ecd7a in term 1.
I20260619 08:18:14.333742 21341 leader_election.cc:302] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e3a91440dee447fbe0370bc875ecd7a, fb7d8cd1362143b8b99c78a8afae9ec3; no voters:
I20260619 08:18:14.333866 21605 raft_consensus.cc:2781] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:14.334036 21605 raft_consensus.cc:686] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 1 LEADER]: Becoming Leader. State: Replica: 1e3a91440dee447fbe0370bc875ecd7a, State: Running, Role: LEADER
I20260619 08:18:14.334137 21605 consensus_queue.cc:237] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } }
I20260619 08:18:14.334925 21135 catalog_manager.cc:5495] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e3a91440dee447fbe0370bc875ecd7a (127.2.119.194). New cstate: current_term: 1 leader_uuid: "1e3a91440dee447fbe0370bc875ecd7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e3a91440dee447fbe0370bc875ecd7a" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 44007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 } health_report { overall_health: UNKNOWN } } }
I20260619 08:18:14.357843 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.355589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.2.119.194:44007)
0619 08:18:14.355649 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:14.355651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:14.355660 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:14.355695 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:14.355696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:14.355714 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:14.355829 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:14.355988 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.356451 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.357178 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:14.357387 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:14.357452 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:14.357575 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:14.357658 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:14.357734 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:14.357754 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 08:18:14.357762 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:18:14.359735 21539 raft_consensus.cc:1261] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 1 FOLLOWER]: Refusing update from remote peer 1e3a91440dee447fbe0370bc875ecd7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:14.359736 21277 raft_consensus.cc:1261] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 1 FOLLOWER]: Refusing update from remote peer 1e3a91440dee447fbe0370bc875ecd7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:14.360177 21609 consensus_queue.cc:1035] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b52358f31eb74ea7a447edc12fd35add" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 42887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:14.360311 21605 consensus_queue.cc:1035] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb7d8cd1362143b8b99c78a8afae9ec3" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:14.362911 21612 mvcc.cc:205] Tried to move back new op lower bound from 7298486658494840832 to 7298486658393161728. Current Snapshot: MvccSnapshot[applied={T|T < 7298486658494840832}]
I20260619 08:18:14.363481 21613 mvcc.cc:205] Tried to move back new op lower bound from 7298486658494840832 to 7298486658393161728. Current Snapshot: MvccSnapshot[applied={T|T < 7298486658494840832}]
I20260619 08:18:14.365520 21134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:14.365597 21134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:14.366389 21134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9ad4212f2784c34a5bfae8d2f4454b4: Sending DeleteTablet for 3 replicas of tablet cb649a1365ae4c30bf31f4c68b5eee87
I20260619 08:18:14.366711 21257 tablet_service.cc:1514] Processing DeleteTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:14 UTC) from {username='slave'} at 127.0.0.1:57238
I20260619 08:18:14.366775 21388 tablet_service.cc:1514] Processing DeleteTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:14 UTC) from {username='slave'} at 127.0.0.1:43832
I20260619 08:18:14.366843 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet cb649a1365ae4c30bf31f4c68b5eee87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:14 UTC) from {username='slave'} at 127.0.0.1:40866
I20260619 08:18:14.367071 21622 tablet_replica.cc:330] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add: stopping tablet replica
I20260619 08:18:14.367084 21624 tablet_replica.cc:330] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: stopping tablet replica
I20260619 08:18:14.367254 21624 raft_consensus.cc:2229] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:14.367254 21622 raft_consensus.cc:2229] T cb649a1365ae4c30bf31f4c68b5eee87 P b52358f31eb74ea7a447edc12fd35add [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:14.367323 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21195
I20260619 08:18:14.367374 21624 raft_consensus.cc:2258] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:14.368613 21623 tablet_replica.cc:330] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: stopping tablet replica
I20260619 08:18:14.368775 21623 raft_consensus.cc:2229] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:18:14.368948 21623 raft_consensus.cc:2258] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:14.369575 21623 ts_tablet_manager.cc:1905] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:14.369940 21624 ts_tablet_manager.cc:1905] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:14.371474 21624 ts_tablet_manager.cc:1918] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:14.371554 21624 log.cc:1196] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/cb649a1365ae4c30bf31f4c68b5eee87
I20260619 08:18:14.371491 21623 ts_tablet_manager.cc:1918] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:14.371711 21623 log.cc:1196] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/cb649a1365ae4c30bf31f4c68b5eee87
I20260619 08:18:14.371816 21624 ts_tablet_manager.cc:1939] T cb649a1365ae4c30bf31f4c68b5eee87 P fb7d8cd1362143b8b99c78a8afae9ec3: Deleting consensus metadata
I20260619 08:18:14.372123 21623 ts_tablet_manager.cc:1939] T cb649a1365ae4c30bf31f4c68b5eee87 P 1e3a91440dee447fbe0370bc875ecd7a: Deleting consensus metadata
I20260619 08:18:14.372399 21119 catalog_manager.cc:4841] TS fb7d8cd1362143b8b99c78a8afae9ec3 (127.2.119.195:35183): tablet cb649a1365ae4c30bf31f4c68b5eee87 (table test-table [id=5f566a980737481192cb60f1c5ad30e4]) successfully deleted
I20260619 08:18:14.372581 21119 catalog_manager.cc:4841] TS 1e3a91440dee447fbe0370bc875ecd7a (127.2.119.194:44007): tablet cb649a1365ae4c30bf31f4c68b5eee87 (table test-table [id=5f566a980737481192cb60f1c5ad30e4]) successfully deleted
W20260619 08:18:14.374130 21121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:42887 (error 108)
I20260619 08:18:14.374338 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21326
I20260619 08:18:14.374979 21597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:14.374516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:42887 (local address 127.0.0.1:57242)
0619 08:18:14.374908 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:14.374947 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:42887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371}
W20260619 08:18:14.375296 21121 catalog_manager.cc:4584] TS b52358f31eb74ea7a447edc12fd35add (127.2.119.193:42887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb649a1365ae4c30bf31f4c68b5eee87: Network error: Client connection negotiation failed: client connection to 127.2.119.193:42887: connect: Connection refused (error 111)
I20260619 08:18:14.379599 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21457
I20260619 08:18:14.384719 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21104
2026-06-19T08:18:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:14 dist-test-slave-d0b5 krb5kdc[21628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:14 dist-test-slave-d0b5 krb5kdc[21628](info): set up 2 sockets
Jun 19 08:18:14 dist-test-slave-d0b5 krb5kdc[21628](info): commencing operation
krb5kdc: starting...
W20260619 08:18:15.434283 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:15.554418 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40501
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:40501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:15.635176 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:15.635407 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:15.635447 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:15.636612 21641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:15.636693 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:15.636719 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:15.636751 21641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:15.636775 21641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:15.638309 21641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:40501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:40501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:15.638578 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:15.638803 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:15.643868 21647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:15.643920 21649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:15.643932 21646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:15.644062 21641 server_base.cc:1034] running on GCE node
I20260619 08:18:15.644343 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:15.644675 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:15.645877 21641 hybrid_clock.cc:648] HybridClock initialized: now 1781857095645840 us; error 53 us; skew 500 ppm
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:15.648587 21641 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:15.648969 21641 webserver.cc:460] Webserver started at http://127.2.119.254:45593/ using document root <none> and password file <none>
I20260619 08:18:15.649161 21641 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:15.649201 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:15.649287 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:15.650348 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "cd53437277ac4f8f9b0f723ec5cd2371"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "6f3644fb5d679138188ba068793ddd7e"
server_key_iv: "b9ae6ad6bd0050707b63e2e65a1331c6"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.650640 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "cd53437277ac4f8f9b0f723ec5cd2371"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "6f3644fb5d679138188ba068793ddd7e"
server_key_iv: "b9ae6ad6bd0050707b63e2e65a1331c6"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.651892 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:18:15.652578 21656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:15.652792 21641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:15.652889 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "cd53437277ac4f8f9b0f723ec5cd2371"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "6f3644fb5d679138188ba068793ddd7e"
server_key_iv: "b9ae6ad6bd0050707b63e2e65a1331c6"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.652972 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:15.664604 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:15.666667 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:15.666841 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:15.670776 21641 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:40501
I20260619 08:18:15.670856 21708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:40501 every 8 connection(s)
I20260619 08:18:15.671196 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:15.672156 21709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:15.674571 21709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: Bootstrap starting.
I20260619 08:18:15.675185 21709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:15.675482 21709 log.cc:824] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:15.676214 21709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: No bootstrap required, opened a new log
I20260619 08:18:15.678005 21709 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } }
I20260619 08:18:15.678141 21709 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:15.678162 21709 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd53437277ac4f8f9b0f723ec5cd2371, State: Initialized, Role: FOLLOWER
I20260619 08:18:15.678326 21709 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } }
I20260619 08:18:15.678407 21709 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:15.678467 21709 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:15.678519 21709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:15.679015 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21641
I20260619 08:18:15.679141 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:15.679180 21709 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } }
I20260619 08:18:15.679293 21709 leader_election.cc:302] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd53437277ac4f8f9b0f723ec5cd2371; no voters:
I20260619 08:18:15.679342 2527 external_mini_cluster.cc:1428] Setting key 451c6ed1774dbb1232a18a425317f754
I20260619 08:18:15.679464 21709 leader_election.cc:288] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:15.679495 21712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:15.679662 21712 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [term 1 LEADER]: Becoming Leader. State: Replica: cd53437277ac4f8f9b0f723ec5cd2371, State: Running, Role: LEADER
I20260619 08:18:15.679718 21709 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:15.679790 21712 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } }
I20260619 08:18:15.680233 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } } }
I20260619 08:18:15.680409 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:15.680377 21713 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd53437277ac4f8f9b0f723ec5cd2371. Latest consensus state: current_term: 1 leader_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd53437277ac4f8f9b0f723ec5cd2371" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 40501 } } }
I20260619 08:18:15.680641 21713 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:15.680816 21723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:15.681411 21723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:15.683020 21723 catalog_manager.cc:1347] Generated new cluster ID: 004da72f4e894671a516c7a80b21fce7
I20260619 08:18:15.683075 21723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:15.690166 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:15.679978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.2.119.254:40501)
0619 08:18:15.680123 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:15.680132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:15.680287 (+ 155us) server_negotiation.cc:407] Connection header received
0619 08:18:15.680406 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:15.680413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:15.680472 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:15.680615 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:15.681790 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.683129 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.683892 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.684322 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.684441 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:15.686886 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:15.686890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:15.686890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:15.686902 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:15.688564 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:15.688948 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:15.688951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:15.688951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:15.689052 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:15.689268 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:15.689269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:15.689269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:15.689550 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:15.689729 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:15.689848 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:18:15.689999 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260619 08:18:15.693246 21723 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:15.693953 21723 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:15.699894 21723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: Generated new TSK 0
I20260619 08:18:15.700080 21723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.119.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
Entry for principal kudu/127.2.119.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab.
I20260619 08:18:15.732594 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:15.814828 21735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:15.815023 21735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:15.815047 21735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:15.816131 21735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:15.816169 21735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:15.816193 21735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:15.817703 21735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.193.keytab
--principal=kudu/127.2.119.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:15.817891 21735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:15.818064 21735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:15.823100 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:15.823132 21735 server_base.cc:1034] running on GCE node
W20260619 08:18:15.823114 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:15.823119 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:15.823509 21735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:15.823755 21735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:15.824920 21735 hybrid_clock.cc:648] HybridClock initialized: now 1781857095824896 us; error 36 us; skew 500 ppm
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:15.827592 21735 init.cc:377] Logged in from keytab as kudu/127.2.119.193@KRBTEST.COM (short username kudu)
I20260619 08:18:15.828068 21735 webserver.cc:460] Webserver started at http://127.2.119.193:44311/ using document root <none> and password file <none>
I20260619 08:18:15.828313 21735 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:15.828383 21735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:15.828541 21735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:15.829655 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "83b7bbf3e87545959bb41e0b40184f19"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "b8643969296b04ddcddfa749dd6d7f7e"
server_key_iv: "61b9f49663328e59eaeef6e2b544824c"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.830015 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "83b7bbf3e87545959bb41e0b40184f19"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "b8643969296b04ddcddfa749dd6d7f7e"
server_key_iv: "61b9f49663328e59eaeef6e2b544824c"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.831321 21735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:15.832129 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:15.832319 21735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:15.832417 21735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "83b7bbf3e87545959bb41e0b40184f19"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "b8643969296b04ddcddfa749dd6d7f7e"
server_key_iv: "61b9f49663328e59eaeef6e2b544824c"
server_key_version: "encryptionkey@0"
I20260619 08:18:15.832499 21735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:15.861884 21735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:15.863986 21735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:15.864158 21735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:15.864408 21735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:15.864732 21735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:15.864790 21735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:15.864827 21735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:15.864876 21735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:15.870770 21735 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:38253
I20260619 08:18:15.870815 21863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:38253 every 8 connection(s)
I20260619 08:18:15.871141 21735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.193@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:15.877825 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21735
I20260619 08:18:15.877975 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:15.878194 2527 external_mini_cluster.cc:1428] Setting key 924e134303412ef7e7f58d63f7475554
I20260619 08:18:15.881443 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:15.871918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:46435 (local address 127.2.119.254:40501)
0619 08:18:15.872040 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:15.872043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:15.872224 (+ 181us) server_negotiation.cc:407] Connection header received
0619 08:18:15.872760 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:15.872761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:15.872775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:15.872827 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:15.873847 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.874444 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.875604 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.875808 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.875864 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:15.878189 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:15.878191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:15.878191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:15.878200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:15.879871 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:15.880328 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:15.880329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:15.880330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:15.880445 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:15.880713 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:15.880714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:15.880715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:15.880856 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:15.880949 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:15.881230 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 08:18:15.881353 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 08:18:15.881669 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:15.871892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:40501 (local address 127.2.119.193:46435)
0619 08:18:15.872095 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:15.872136 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:15.872671 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:15.872887 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:15.872890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:15.873544 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:15.873760 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:15.873765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.874534 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.874534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:15.875528 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:15.875530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:15.875885 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:15.875886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:15.875998 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:15.876493 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:15.876502 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:15.878090 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:15.880010 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:15.880018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:15.880020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:15.880248 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:15.880525 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:15.880526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:15.880526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:15.880637 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:15.880942 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:15.880949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:15.881091 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 08:18:15.881197 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:18:15.881378 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260619 08:18:15.882171 21864 heartbeater.cc:347] Connected to a master server at 127.2.119.254:40501
I20260619 08:18:15.882261 21864 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:15.882457 21864 heartbeater.cc:511] Master 127.2.119.254:40501 requested a full tablet report, sending...
I20260619 08:18:15.883030 21673 ts_manager.cc:194] Registered new tserver with Master: 83b7bbf3e87545959bb41e0b40184f19 (127.2.119.193:38253)
I20260619 08:18:15.884178 21673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.193@KRBTEST.COM'} at 127.2.119.193:46435
WARNING: no policy specified for kudu/127.2.119.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
Entry for principal kudu/127.2.119.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab.
I20260619 08:18:15.906047 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:15.988313 21869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:15.988524 21869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:15.988548 21869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:15.989710 21869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:15.989754 21869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:15.989781 21869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:15.991180 21869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.194.keytab
--principal=kudu/127.2.119.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:15.991372 21869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:15.991542 21869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:15.996690 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:15.996709 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:15.996789 21869 server_base.cc:1034] running on GCE node
W20260619 08:18:15.996871 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:15.997238 21869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:15.997520 21869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:15.998670 21869 hybrid_clock.cc:648] HybridClock initialized: now 1781857095998642 us; error 52 us; skew 500 ppm
Jun 19 08:18:15 dist-test-slave-d0b5 krb5kdc[21628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:16.001253 21869 init.cc:377] Logged in from keytab as kudu/127.2.119.194@KRBTEST.COM (short username kudu)
I20260619 08:18:16.001713 21869 webserver.cc:460] Webserver started at http://127.2.119.194:45011/ using document root <none> and password file <none>
I20260619 08:18:16.001909 21869 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:16.001950 21869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:16.002035 21869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:16.003031 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "91e2485b2c014ac7a75a0bf0537ab11b"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "bbda73e39adfa729ff72094c5a2cf5e4"
server_key_iv: "d90cf7c8bbcf49b103800cbd9b9c38a5"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.003391 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "91e2485b2c014ac7a75a0bf0537ab11b"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "bbda73e39adfa729ff72094c5a2cf5e4"
server_key_iv: "d90cf7c8bbcf49b103800cbd9b9c38a5"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.004666 21869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 08:18:16.005412 21884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.005599 21869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:16.005687 21869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "91e2485b2c014ac7a75a0bf0537ab11b"
format_stamp: "Formatted at 2026-06-19 08:18:15 on dist-test-slave-d0b5"
server_key: "bbda73e39adfa729ff72094c5a2cf5e4"
server_key_iv: "d90cf7c8bbcf49b103800cbd9b9c38a5"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.005760 21869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:16.052893 21869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:16.055311 21869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:16.055477 21869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:16.055692 21869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:16.056037 21869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:16.056074 21869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.056097 21869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:16.056146 21869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.062002 21869 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:46165
I20260619 08:18:16.062050 21997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:46165 every 8 connection(s)
I20260619 08:18:16.062364 21869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
Jun 19 08:18:16 dist-test-slave-d0b5 krb5kdc[21628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857095, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.194@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:16.071544 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 21869
I20260619 08:18:16.071666 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:16.071839 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.063138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:45673 (local address 127.2.119.254:40501)
0619 08:18:16.063190 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.063192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.063402 (+ 210us) server_negotiation.cc:407] Connection header received
0619 08:18:16.063929 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.063931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.063952 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.064009 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:16.064954 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.065512 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.066510 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.066708 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.066783 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.069133 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:16.069134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:16.069134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:16.069141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:16.070504 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.070919 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.070919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.070920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.070974 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.071199 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.071199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.071200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.071300 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:16.071374 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.071653 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.071739 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:18:16.071882 2527 external_mini_cluster.cc:1428] Setting key 91f059c9b0f58d03d55823667006dfce
I20260619 08:18:16.072124 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.063117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:40501 (local address 127.2.119.194:45673)
0619 08:18:16.063283 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.063319 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.063822 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.064078 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.064082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.064658 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:16.064846 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.064850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.065605 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.065606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.066403 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.066406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.066789 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.066791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.066922 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.067424 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:16.067432 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:16.069034 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:16.070587 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:16.070593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:16.070595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:16.070848 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:16.071036 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:16.071037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:16.071037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:16.071140 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:16.071381 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:16.071387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:16.071507 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.071603 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.071813 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260619 08:18:16.072724 21998 heartbeater.cc:347] Connected to a master server at 127.2.119.254:40501
I20260619 08:18:16.072832 21998 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:16.073079 21998 heartbeater.cc:511] Master 127.2.119.254:40501 requested a full tablet report, sending...
I20260619 08:18:16.073520 21672 ts_manager.cc:194] Registered new tserver with Master: 91e2485b2c014ac7a75a0bf0537ab11b (127.2.119.194:46165)
I20260619 08:18:16.073931 21672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.194@KRBTEST.COM'} at 127.2.119.194:45673
WARNING: no policy specified for kudu/127.2.119.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
Entry for principal kudu/127.2.119.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab.
I20260619 08:18:16.098050 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:16.178720 22003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:16.178925 22003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:16.178948 22003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:16.180182 22003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:16.180263 22003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:16.180310 22003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:16.181974 22003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.195.keytab
--principal=kudu/127.2.119.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:40501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:16.182148 22003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:16.182317 22003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:16.187218 22009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:16.187208 22011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:16.187527 22008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:16.187492 22003 server_base.cc:1034] running on GCE node
I20260619 08:18:16.187716 22003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:16.187916 22003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:16.189051 22003 hybrid_clock.cc:648] HybridClock initialized: now 1781857096189041 us; error 26 us; skew 500 ppm
Jun 19 08:18:16 dist-test-slave-d0b5 krb5kdc[21628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857096, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:16.191519 22003 init.cc:377] Logged in from keytab as kudu/127.2.119.195@KRBTEST.COM (short username kudu)
I20260619 08:18:16.191972 22003 webserver.cc:460] Webserver started at http://127.2.119.195:46061/ using document root <none> and password file <none>
I20260619 08:18:16.192169 22003 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:16.192211 22003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:16.192297 22003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:16.193270 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "2192462496284bab9d621e4cd61e7abe"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "d5c155a3004cb9486a92bd7fa87647ef"
server_key_iv: "55d57a71d1fa007d603bed3696d39c56"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.193636 22003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "2192462496284bab9d621e4cd61e7abe"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "d5c155a3004cb9486a92bd7fa87647ef"
server_key_iv: "55d57a71d1fa007d603bed3696d39c56"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.194837 22003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:16.195539 22018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.195711 22003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:16.195820 22003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "2192462496284bab9d621e4cd61e7abe"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "d5c155a3004cb9486a92bd7fa87647ef"
server_key_iv: "55d57a71d1fa007d603bed3696d39c56"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.195945 22003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:16.235891 22003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:16.238359 22003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:16.238526 22003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:16.238780 22003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:16.239096 22003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:16.239151 22003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.239195 22003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:16.239218 22003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.245119 22003 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:37359
I20260619 08:18:16.245163 22131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:37359 every 8 connection(s)
I20260619 08:18:16.245543 22003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
Jun 19 08:18:16 dist-test-slave-d0b5 krb5kdc[21628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857096, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.195@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:16.253747 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22003
I20260619 08:18:16.253875 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:16.254076 2527 external_mini_cluster.cc:1428] Setting key ffeb7f892a66936240b89755825c6dc5
I20260619 08:18:16.255371 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.246410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:40629 (local address 127.2.119.254:40501)
0619 08:18:16.246570 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.246573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.246739 (+ 166us) server_negotiation.cc:407] Connection header received
0619 08:18:16.247379 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.247381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.247395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.247478 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:16.248469 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.248982 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.250042 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.250245 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.250332 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.252411 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:16.252412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:16.252413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:16.252421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:16.253789 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.254328 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.254330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.254330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.254402 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.254684 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.254685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.254685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.254821 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:16.254943 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.255237 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.255302 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260619 08:18:16.255649 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.246327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:40501 (local address 127.2.119.195:40629)
0619 08:18:16.246593 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.246643 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.247227 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.247563 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.247567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.248147 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:16.248356 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.248362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.249078 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.249079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.249942 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.249944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.250341 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.250342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.250474 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.250950 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:16.250958 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:16.252303 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:16.253937 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:16.253944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:16.253947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:16.254252 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:16.254478 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:16.254479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:16.254480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:16.254616 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:16.254958 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:16.254964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:16.255118 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.255204 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.255381 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":91,"threads_started":1}
I20260619 08:18:16.256194 22132 heartbeater.cc:347] Connected to a master server at 127.2.119.254:40501
I20260619 08:18:16.256275 22132 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:16.256479 22132 heartbeater.cc:511] Master 127.2.119.254:40501 requested a full tablet report, sending...
I20260619 08:18:16.256861 21672 ts_manager.cc:194] Registered new tserver with Master: 2192462496284bab9d621e4cd61e7abe (127.2.119.195:37359)
I20260619 08:18:16.257216 21672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.119.195@KRBTEST.COM'} at 127.2.119.195:40629
I20260619 08:18:16.265882 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:16.273229 21716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.267264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46528 (local address 127.2.119.254:40501)
0619 08:18:16.267330 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.267332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.267406 (+ 74us) server_negotiation.cc:407] Connection header received
0619 08:18:16.267446 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.267446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.267455 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.267506 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:16.268081 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.268643 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.269482 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.269674 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.269732 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.270939 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:16.270940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:16.270941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:16.270947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:16.272114 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.272447 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.272447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.272448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.272496 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:16.272733 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:16.272733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:16.272734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:16.272899 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:16.272976 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.273105 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.273165 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:18:16.275223 21672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:16.277051 21672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:16.284202 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57516 (local address 127.2.119.193:38253)
0619 08:18:16.279875 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.279881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.280035 (+ 154us) server_negotiation.cc:407] Connection header received
0619 08:18:16.280187 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.280190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.280203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.280292 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:16.280497 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.281586 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.282970 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.283739 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283837 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.283901 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.284060 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.284102 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260619 08:18:16.284229 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:46165 (local address 127.0.0.1:35250)
0619 08:18:16.279952 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.279961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.280026 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.280509 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.280510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.280512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:16.280563 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.280564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.282090 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.282091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.283105 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.283106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.283973 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.284033 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.284038 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.284086 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.284127 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":62,"threads_started":1}
I20260619 08:18:16.284250 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:38253 (local address 127.0.0.1:57516)
0619 08:18:16.279935 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.279971 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.280067 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.280312 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.280315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.280317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:16.280394 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.280396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.281689 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.281690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.282878 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.282880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.283839 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.283929 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.283943 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.284042 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.284101 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":116,"threads_started":1}
I20260619 08:18:16.284548 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.2.119.194:46165)
0619 08:18:16.280132 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.280136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.280146 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:16.280186 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.280188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.280232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.280440 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:16.280625 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.281981 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283195 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.283820 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283888 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.283938 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.284417 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.284466 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260619 08:18:16.284603 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:37359 (local address 127.0.0.1:34706)
0619 08:18:16.279868 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.279961 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.280026 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.280915 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.280916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.280918 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:16.280956 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.280957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.282366 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.282367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.283418 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.283421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.284377 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.284378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.284437 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.284440 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.284495 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.284529 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":66,"threads_started":1}
I20260619 08:18:16.284672 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.279620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.2.119.195:37359)
0619 08:18:16.280639 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.280644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.280655 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:16.280697 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.280699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.280716 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.280891 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:16.281021 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.282250 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.283517 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.284225 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.284286 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.284340 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.284546 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.284592 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":108,"threads_started":1}
I20260619 08:18:16.285375 22066 tablet_service.cc:1467] Processing CreateTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab (DEFAULT_TABLE table=test-table [id=8630eb7c9ec64844a65e95c32beb36f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:16.285368 21931 tablet_service.cc:1467] Processing CreateTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab (DEFAULT_TABLE table=test-table [id=8630eb7c9ec64844a65e95c32beb36f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:16.285713 21931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53d3878f6e44df2bee10c1c20bee9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:16.285713 22066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53d3878f6e44df2bee10c1c20bee9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:16.285876 21798 tablet_service.cc:1467] Processing CreateTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab (DEFAULT_TABLE table=test-table [id=8630eb7c9ec64844a65e95c32beb36f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:16.286129 21798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a53d3878f6e44df2bee10c1c20bee9ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:16.288882 22149 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Bootstrap starting.
I20260619 08:18:16.289049 22150 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Bootstrap starting.
I20260619 08:18:16.289412 22151 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Bootstrap starting.
I20260619 08:18:16.289572 22149 tablet_bootstrap.cc:654] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:16.289692 22150 tablet_bootstrap.cc:654] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:16.290124 22149 log.cc:824] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:16.290143 22151 tablet_bootstrap.cc:654] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:16.290261 22150 log.cc:824] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:16.290731 22151 log.cc:824] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:16.291018 22149 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: No bootstrap required, opened a new log
I20260619 08:18:16.291091 22150 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: No bootstrap required, opened a new log
I20260619 08:18:16.291103 22149 ts_tablet_manager.cc:1397] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:16.291245 22150 ts_tablet_manager.cc:1397] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 08:18:16.291501 22151 tablet_bootstrap.cc:492] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: No bootstrap required, opened a new log
I20260619 08:18:16.291620 22151 ts_tablet_manager.cc:1397] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:16.292827 22150 raft_consensus.cc:348] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.293007 22150 raft_consensus.cc:374] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:16.293076 22150 raft_consensus.cc:729] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e2485b2c014ac7a75a0bf0537ab11b, State: Initialized, Role: FOLLOWER
I20260619 08:18:16.293072 22149 raft_consensus.cc:348] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.293249 22149 raft_consensus.cc:374] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:16.293213 22150 consensus_queue.cc:260] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.293314 22149 raft_consensus.cc:729] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b7bbf3e87545959bb41e0b40184f19, State: Initialized, Role: FOLLOWER
I20260619 08:18:16.293422 22149 consensus_queue.cc:260] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.293583 21998 heartbeater.cc:503] Master 127.2.119.254:40501 was elected leader, sending a full tablet report...
I20260619 08:18:16.293682 22149 ts_tablet_manager.cc:1428] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:16.293704 21864 heartbeater.cc:503] Master 127.2.119.254:40501 was elected leader, sending a full tablet report...
I20260619 08:18:16.293864 22150 ts_tablet_manager.cc:1428] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:16.293529 22151 raft_consensus.cc:348] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.293941 22151 raft_consensus.cc:374] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:16.293996 22151 raft_consensus.cc:729] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2192462496284bab9d621e4cd61e7abe, State: Initialized, Role: FOLLOWER
I20260619 08:18:16.294147 22151 consensus_queue.cc:260] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.294440 22151 ts_tablet_manager.cc:1428] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 08:18:16.294581 22132 heartbeater.cc:503] Master 127.2.119.254:40501 was elected leader, sending a full tablet report...
W20260619 08:18:16.312940 21999 tablet.cc:2369] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:16.372198 21865 tablet.cc:2369] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 08:18:16.496204 22133 tablet.cc:2369] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:16.504257 22157 raft_consensus.cc:482] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:16.504436 22157 raft_consensus.cc:504] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.504849 22157 leader_election.cc:288] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83b7bbf3e87545959bb41e0b40184f19 (127.2.119.193:38253), 91e2485b2c014ac7a75a0bf0537ab11b (127.2.119.194:46165)
I20260619 08:18:16.507905 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.505225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:36179 (local address 127.2.119.194:46165)
0619 08:18:16.505338 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.505340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.505351 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:16.505384 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.505384 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.505394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.505469 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:16.505632 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.506236 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507191 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.507554 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507639 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.507648 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.507795 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.507831 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:18:16.507905 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.505117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:46165 (local address 127.2.119.195:36179)
0619 08:18:16.505174 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.505187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.505237 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.505465 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.505466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.505469 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:16.505553 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.505555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.506320 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.506321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.507088 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.507091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.507636 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.507720 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.507723 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.507790 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.507831 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260619 08:18:16.508215 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.505161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:49129 (local address 127.2.119.193:38253)
0619 08:18:16.505218 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.505220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.505582 (+ 362us) server_negotiation.cc:407] Connection header received
0619 08:18:16.505707 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.505708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.505719 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.505767 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:16.505997 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.506493 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507418 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.507795 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507881 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.507889 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.508111 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.508154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:18:16.508457 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.505120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:38253 (local address 127.2.119.195:49129)
0619 08:18:16.505525 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.505535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.505588 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.505784 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.505785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.505787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:16.505865 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.505867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.506592 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.506593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.507321 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.507324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.507892 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.507894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.508003 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.508009 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.508338 (+ 329us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.508386 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":212,"thread_start_us":130,"threads_started":1}
I20260619 08:18:16.508438 21952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53d3878f6e44df2bee10c1c20bee9ab" candidate_uuid: "2192462496284bab9d621e4cd61e7abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" is_pre_election: true
I20260619 08:18:16.508597 21952 raft_consensus.cc:2454] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2192462496284bab9d621e4cd61e7abe in term 0.
I20260619 08:18:16.508826 22021 leader_election.cc:302] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2192462496284bab9d621e4cd61e7abe, 91e2485b2c014ac7a75a0bf0537ab11b; no voters:
I20260619 08:18:16.508832 21818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53d3878f6e44df2bee10c1c20bee9ab" candidate_uuid: "2192462496284bab9d621e4cd61e7abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b7bbf3e87545959bb41e0b40184f19" is_pre_election: true
I20260619 08:18:16.508946 22157 raft_consensus.cc:2781] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:16.509001 22157 raft_consensus.cc:482] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:16.509006 21818 raft_consensus.cc:2454] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2192462496284bab9d621e4cd61e7abe in term 0.
I20260619 08:18:16.509037 22157 raft_consensus.cc:3037] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:16.509869 22157 raft_consensus.cc:504] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.510032 22157 leader_election.cc:288] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [CANDIDATE]: Term 1 election: Requested vote from peers 83b7bbf3e87545959bb41e0b40184f19 (127.2.119.193:38253), 91e2485b2c014ac7a75a0bf0537ab11b (127.2.119.194:46165)
I20260619 08:18:16.510188 21818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53d3878f6e44df2bee10c1c20bee9ab" candidate_uuid: "2192462496284bab9d621e4cd61e7abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b7bbf3e87545959bb41e0b40184f19"
I20260619 08:18:16.510236 21952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a53d3878f6e44df2bee10c1c20bee9ab" candidate_uuid: "2192462496284bab9d621e4cd61e7abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e2485b2c014ac7a75a0bf0537ab11b"
I20260619 08:18:16.510273 21818 raft_consensus.cc:3037] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:16.510334 21952 raft_consensus.cc:3037] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:16.511031 21818 raft_consensus.cc:2454] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2192462496284bab9d621e4cd61e7abe in term 1.
I20260619 08:18:16.511047 21952 raft_consensus.cc:2454] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2192462496284bab9d621e4cd61e7abe in term 1.
I20260619 08:18:16.511207 22019 leader_election.cc:302] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2192462496284bab9d621e4cd61e7abe, 83b7bbf3e87545959bb41e0b40184f19; no voters:
I20260619 08:18:16.511317 22157 raft_consensus.cc:2781] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:16.511462 22157 raft_consensus.cc:686] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 1 LEADER]: Becoming Leader. State: Replica: 2192462496284bab9d621e4cd61e7abe, State: Running, Role: LEADER
I20260619 08:18:16.511562 22157 consensus_queue.cc:237] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } }
I20260619 08:18:16.512480 21673 catalog_manager.cc:5495] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe reported cstate change: term changed from 0 to 1, leader changed from <none> to 2192462496284bab9d621e4cd61e7abe (127.2.119.195). New cstate: current_term: 1 leader_uuid: "2192462496284bab9d621e4cd61e7abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2192462496284bab9d621e4cd61e7abe" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 37359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 } health_report { overall_health: UNKNOWN } } }
I20260619 08:18:16.566793 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.564248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.2.119.195:37359)
0619 08:18:16.564318 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.564320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.564337 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:18:16.564372 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.564373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.564389 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.564465 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:16.564639 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.565102 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.565949 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.566158 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.566232 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.566392 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:16.566497 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:16.566584 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.566667 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.566686 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:18:16.568738 21818 raft_consensus.cc:1261] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 1 FOLLOWER]: Refusing update from remote peer 2192462496284bab9d621e4cd61e7abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:16.568739 21952 raft_consensus.cc:1261] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 1 FOLLOWER]: Refusing update from remote peer 2192462496284bab9d621e4cd61e7abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:16.569126 22159 consensus_queue.cc:1035] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e2485b2c014ac7a75a0bf0537ab11b" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 46165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:16.569250 22157 consensus_queue.cc:1035] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b7bbf3e87545959bb41e0b40184f19" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:16.571583 22162 mvcc.cc:205] Tried to move back new op lower bound from 7298486667542618112 to 7298486667311976448. Current Snapshot: MvccSnapshot[applied={T|T < 7298486667542618112}]
I20260619 08:18:16.571763 22164 mvcc.cc:205] Tried to move back new op lower bound from 7298486667542618112 to 7298486667311976448. Current Snapshot: MvccSnapshot[applied={T|T < 7298486667542618112}]
I20260619 08:18:16.571795 22163 mvcc.cc:205] Tried to move back new op lower bound from 7298486667542618112 to 7298486667311976448. Current Snapshot: MvccSnapshot[applied={T|T < 7298486667542618112}]
I20260619 08:18:16.574437 21673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:16.574534 21673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:16.575307 21673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd53437277ac4f8f9b0f723ec5cd2371: Sending DeleteTablet for 3 replicas of tablet a53d3878f6e44df2bee10c1c20bee9ab
I20260619 08:18:16.575664 22066 tablet_service.cc:1514] Processing DeleteTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:16 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:34706
I20260619 08:18:16.575662 21931 tablet_service.cc:1514] Processing DeleteTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:16 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:35250
I20260619 08:18:16.575675 21798 tablet_service.cc:1514] Processing DeleteTablet for tablet a53d3878f6e44df2bee10c1c20bee9ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:16 UTC) from {username='kudu', principal='kudu/127.2.119.254@KRBTEST.COM'} at 127.0.0.1:57516
I20260619 08:18:16.575888 22173 tablet_replica.cc:330] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: stopping tablet replica
I20260619 08:18:16.575989 22173 raft_consensus.cc:2229] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:18:16.576184 22173 raft_consensus.cc:2258] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:16.576274 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21735
I20260619 08:18:16.576350 22174 tablet_replica.cc:330] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19: stopping tablet replica
I20260619 08:18:16.576436 22174 raft_consensus.cc:2229] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:16.576541 22174 raft_consensus.cc:2258] T a53d3878f6e44df2bee10c1c20bee9ab P 83b7bbf3e87545959bb41e0b40184f19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:16.577206 22173 ts_tablet_manager.cc:1905] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:16.578893 22172 tablet_replica.cc:330] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: stopping tablet replica
I20260619 08:18:16.579020 22172 raft_consensus.cc:2229] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:16.579129 22172 raft_consensus.cc:2258] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:16.579324 22173 ts_tablet_manager.cc:1918] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:16.579442 22173 log.cc:1196] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/a53d3878f6e44df2bee10c1c20bee9ab
I20260619 08:18:16.579598 22172 ts_tablet_manager.cc:1905] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:16.579744 22173 ts_tablet_manager.cc:1939] T a53d3878f6e44df2bee10c1c20bee9ab P 2192462496284bab9d621e4cd61e7abe: Deleting consensus metadata
I20260619 08:18:16.580155 21659 catalog_manager.cc:4841] TS 2192462496284bab9d621e4cd61e7abe (127.2.119.195:37359): tablet a53d3878f6e44df2bee10c1c20bee9ab (table test-table [id=8630eb7c9ec64844a65e95c32beb36f7]) successfully deleted
I20260619 08:18:16.581550 22172 ts_tablet_manager.cc:1918] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:16.581624 22172 log.cc:1196] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/a53d3878f6e44df2bee10c1c20bee9ab
I20260619 08:18:16.581912 22172 ts_tablet_manager.cc:1939] T a53d3878f6e44df2bee10c1c20bee9ab P 91e2485b2c014ac7a75a0bf0537ab11b: Deleting consensus metadata
I20260619 08:18:16.582340 21659 catalog_manager.cc:4841] TS 91e2485b2c014ac7a75a0bf0537ab11b (127.2.119.194:46165): tablet a53d3878f6e44df2bee10c1c20bee9ab (table test-table [id=8630eb7c9ec64844a65e95c32beb36f7]) successfully deleted
W20260619 08:18:16.583766 21657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:38253 (error 108)
I20260619 08:18:16.584036 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21869
I20260619 08:18:16.584622 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.584107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:38253 (local address 127.0.0.1:57522)
0619 08:18:16.584551 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.584588 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:38253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425}
W20260619 08:18:16.584751 21657 catalog_manager.cc:4584] TS 83b7bbf3e87545959bb41e0b40184f19 (127.2.119.193:38253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a53d3878f6e44df2bee10c1c20bee9ab: Network error: Client connection negotiation failed: client connection to 127.2.119.193:38253: connect: Connection refused (error 111)
I20260619 08:18:16.590039 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22003
I20260619 08:18:16.594069 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 21641
2026-06-19T08:18:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2210 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T08:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:16Z Disabled control of system clock
I20260619 08:18:16.621285 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42945
--webserver_interface=127.2.119.254
--webserver_port=0
--builtin_ntp_servers=127.2.119.212:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:42945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:16.702612 22179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:16.702859 22179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:16.702881 22179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:16.704094 22179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:16.704139 22179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:16.704152 22179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:16.704164 22179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:16.704176 22179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:16.706102 22179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:42945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:42945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:16.706589 22179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:16.706998 22179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:16.712690 22184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:16.712836 22179 server_base.cc:1034] running on GCE node
W20260619 08:18:16.712641 22185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:16.712643 22187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:16.713493 22179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:16.713833 22179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:16.715011 22179 hybrid_clock.cc:648] HybridClock initialized: now 1781857096714991 us; error 105 us; skew 500 ppm
I20260619 08:18:16.716363 22179 webserver.cc:460] Webserver started at http://127.2.119.254:46489/ using document root <none> and password file <none>
I20260619 08:18:16.716632 22179 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:16.716684 22179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:16.716820 22179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:16.718021 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "f3c2340713f04605ad011a8d23500a44"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "056b75b60be31021cf6442e2497cd6ea"
server_key_iv: "8f10ac6089d97753c533e38876da07d7"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.718412 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "f3c2340713f04605ad011a8d23500a44"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "056b75b60be31021cf6442e2497cd6ea"
server_key_iv: "8f10ac6089d97753c533e38876da07d7"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.719822 22179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:16.721081 22193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.721532 22179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:16.721699 22179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "f3c2340713f04605ad011a8d23500a44"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "056b75b60be31021cf6442e2497cd6ea"
server_key_iv: "8f10ac6089d97753c533e38876da07d7"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.721823 22179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:16.732792 22179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:16.733165 22179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:16.733424 22179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:16.737553 22179 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:42945
I20260619 08:18:16.737636 22245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:42945 every 8 connection(s)
I20260619 08:18:16.737973 22179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:16.738942 22246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:16.741207 22246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: Bootstrap starting.
I20260619 08:18:16.741822 22246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:16.742071 22246 log.cc:824] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:16.742731 22246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: No bootstrap required, opened a new log
I20260619 08:18:16.744285 22246 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } }
I20260619 08:18:16.744434 22246 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:16.744457 22246 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c2340713f04605ad011a8d23500a44, State: Initialized, Role: FOLLOWER
I20260619 08:18:16.744597 22246 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } }
I20260619 08:18:16.744676 22246 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:16.744699 22246 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:16.744727 22246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:16.745208 22246 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } }
I20260619 08:18:16.745339 22246 leader_election.cc:302] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3c2340713f04605ad011a8d23500a44; no voters:
I20260619 08:18:16.745527 22246 leader_election.cc:288] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:16.745559 22249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:16.745682 22249 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [term 1 LEADER]: Becoming Leader. State: Replica: f3c2340713f04605ad011a8d23500a44, State: Running, Role: LEADER
I20260619 08:18:16.745764 22249 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } }
I20260619 08:18:16.745810 22246 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:16.746028 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22179
I20260619 08:18:16.746084 22249 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3c2340713f04605ad011a8d23500a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } } }
I20260619 08:18:16.746161 22249 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:16.746301 22249 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3c2340713f04605ad011a8d23500a44. Latest consensus state: current_term: 1 leader_uuid: "f3c2340713f04605ad011a8d23500a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3c2340713f04605ad011a8d23500a44" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 42945 } } }
I20260619 08:18:16.746366 22249 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:16.746305 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:16.746635 2527 external_mini_cluster.cc:1428] Setting key 2f415f9c21c93a0be54e68c86356fcc0
W20260619 08:18:16.747828 22266 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:16.747915 22266 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:16.748005 22254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:16.748250 22254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:16.749719 22254 catalog_manager.cc:1347] Generated new cluster ID: 48d4fc6eb51d45738ee82e393394bf37
I20260619 08:18:16.749773 22254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:16.752952 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.747360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49896 (local address 127.2.119.254:42945)
0619 08:18:16.747665 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.747675 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.747708 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:18:16.747829 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.747837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.747912 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.748063 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:16.748283 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.749647 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.750771 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.751370 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.751492 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.752237 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:16.752240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:16.752241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:16.752448 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:16.752487 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:16.752501 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:16.752564 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.752665 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.752742 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260619 08:18:16.770778 22254 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:16.771389 22254 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:16.780543 22254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: Generated new TSK 0
I20260619 08:18:16.780746 22254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:16.786733 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.193:0
--local_ip_for_outbound_sockets=127.2.119.193
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--builtin_ntp_servers=127.2.119.212:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:16.893205 22270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:16.893460 22270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:16.893496 22270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:16.894701 22270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:16.894770 22270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:16.894817 22270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.193
I20260619 08:18:16.896261 22270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.119.193
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.193
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:16.896538 22270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:16.896839 22270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:16.902920 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:16.902935 22276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:16.903205 22270 server_base.cc:1034] running on GCE node
W20260619 08:18:16.903208 22278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:16.903483 22270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:16.903738 22270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:16.904933 22270 hybrid_clock.cc:648] HybridClock initialized: now 1781857096904898 us; error 35 us; skew 500 ppm
I20260619 08:18:16.906466 22270 webserver.cc:460] Webserver started at http://127.2.119.193:42279/ using document root <none> and password file <none>
I20260619 08:18:16.906724 22270 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:16.906782 22270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:16.906916 22270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:16.908299 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data/instance:
uuid: "181943fb328547e4961ff60f92173df2"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "34a53884f2fb562f2ff16aad110636e8"
server_key_iv: "baf2bd50950eb1847fd7573a6291c7b0"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.908739 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance:
uuid: "181943fb328547e4961ff60f92173df2"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "34a53884f2fb562f2ff16aad110636e8"
server_key_iv: "baf2bd50950eb1847fd7573a6291c7b0"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.910286 22270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 08:18:16.911116 22284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.911386 22270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:16.911484 22270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal
uuid: "181943fb328547e4961ff60f92173df2"
format_stamp: "Formatted at 2026-06-19 08:18:16 on dist-test-slave-d0b5"
server_key: "34a53884f2fb562f2ff16aad110636e8"
server_key_iv: "baf2bd50950eb1847fd7573a6291c7b0"
server_key_version: "encryptionkey@0"
I20260619 08:18:16.911557 22270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:16.930776 22270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:16.931133 22270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:16.931298 22270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:16.931550 22270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:16.931901 22270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:16.931934 22270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.931957 22270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:16.932003 22270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:16.938022 22270 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.193:34239
I20260619 08:18:16.938155 22397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.193:34239 every 8 connection(s)
I20260619 08:18:16.938493 22270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/data/info.pb
I20260619 08:18:16.943210 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22270
I20260619 08:18:16.943346 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-0/wal/instance
I20260619 08:18:16.943524 2527 external_mini_cluster.cc:1428] Setting key 1e8f12aed8d17c0505db40873b2c1cc2
I20260619 08:18:16.944412 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.939439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.193:43857 (local address 127.2.119.254:42945)
0619 08:18:16.939516 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:16.939519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:16.939653 (+ 134us) server_negotiation.cc:407] Connection header received
0619 08:18:16.940147 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:16.940148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:16.940162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:16.940232 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:16.940999 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.941568 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.942621 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.942818 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.942938 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.943981 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:16.943982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:16.943983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:16.943988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:16.944003 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:16.944007 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:16.944083 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:16.944287 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 08:18:16.944320 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:18:16.944525 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:16.939320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42945 (local address 127.2.119.193:43857)
0619 08:18:16.939513 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:16.939556 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:16.940052 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:16.940308 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:16.940311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:16.940320 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:16.940814 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.940818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.941675 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.941676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.942509 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:16.942511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:16.942976 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:16.942978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:16.943104 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:16.943764 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:16.943764 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:16.943804 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:16.943834 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:16.944083 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:16.944091 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:16.944094 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:16.944163 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 08:18:16.944285 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260619 08:18:16.944568 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.194:0
--local_ip_for_outbound_sockets=127.2.119.194
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--builtin_ntp_servers=127.2.119.212:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 08:18:16.945192 22398 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42945
I20260619 08:18:16.945335 22398 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:16.945546 22398 heartbeater.cc:511] Master 127.2.119.254:42945 requested a full tablet report, sending...
I20260619 08:18:16.946072 22210 ts_manager.cc:194] Registered new tserver with Master: 181943fb328547e4961ff60f92173df2 (127.2.119.193:34239)
I20260619 08:18:16.946892 22210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.193:43857
W20260619 08:18:17.027333 22401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:17.027536 22401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:17.027558 22401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:17.028717 22401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:17.028759 22401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:17.028784 22401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.194
I20260619 08:18:17.030236 22401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.119.194
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.194
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:17.030494 22401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:17.030804 22401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:17.036181 22407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:17.036237 22406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:17.036190 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:17.036286 22401 server_base.cc:1034] running on GCE node
I20260619 08:18:17.036618 22401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:17.036885 22401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:17.038060 22401 hybrid_clock.cc:648] HybridClock initialized: now 1781857097038053 us; error 35 us; skew 500 ppm
I20260619 08:18:17.039587 22401 webserver.cc:460] Webserver started at http://127.2.119.194:39545/ using document root <none> and password file <none>
I20260619 08:18:17.039861 22401 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:17.039942 22401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:17.040113 22401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:17.042011 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data/instance:
uuid: "24724fb5239a456e9592bee61d4bcaf1"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "8e7cd0dfc94b72a732fa83d346fb7ca7"
server_key_iv: "35ec902edf8c7bb6e4079cc27d8e135b"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.042424 22401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance:
uuid: "24724fb5239a456e9592bee61d4bcaf1"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "8e7cd0dfc94b72a732fa83d346fb7ca7"
server_key_iv: "35ec902edf8c7bb6e4079cc27d8e135b"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.043850 22401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:17.044691 22415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.044950 22401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:17.045059 22401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal
uuid: "24724fb5239a456e9592bee61d4bcaf1"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "8e7cd0dfc94b72a732fa83d346fb7ca7"
server_key_iv: "35ec902edf8c7bb6e4079cc27d8e135b"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.045154 22401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:17.053838 22401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:17.054119 22401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:17.054262 22401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:17.054474 22401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:17.054800 22401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:17.054860 22401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.054903 22401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:17.054929 22401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.060837 22401 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.194:33635
I20260619 08:18:17.060897 22528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.194:33635 every 8 connection(s)
I20260619 08:18:17.061249 22401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/data/info.pb
I20260619 08:18:17.066352 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.062004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:52705 (local address 127.2.119.254:42945)
0619 08:18:17.062051 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.062054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.062258 (+ 204us) server_negotiation.cc:407] Connection header received
0619 08:18:17.062745 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.062746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.062754 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.062815 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:17.063491 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.063999 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.064980 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.065159 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.065212 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.066126 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:17.066128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:17.066128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:17.066132 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:17.066147 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:17.066151 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:17.066195 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.066267 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.066290 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:17.066573 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.061957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42945 (local address 127.2.119.194:52705)
0619 08:18:17.062141 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.062183 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.062663 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.062898 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.062901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.062909 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:17.063416 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.063420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.064090 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.064091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.064903 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.064905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.065267 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.065268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.065494 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.065997 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:17.065997 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:17.066025 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:17.066048 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:17.066206 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:17.066210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:17.066212 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.066260 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.066358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 08:18:17.067166 22529 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42945
I20260619 08:18:17.067270 22529 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:17.067531 22529 heartbeater.cc:511] Master 127.2.119.254:42945 requested a full tablet report, sending...
I20260619 08:18:17.068188 22210 ts_manager.cc:194] Registered new tserver with Master: 24724fb5239a456e9592bee61d4bcaf1 (127.2.119.194:33635)
I20260619 08:18:17.068535 22210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.194:52705
I20260619 08:18:17.070022 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22401
I20260619 08:18:17.070113 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal/instance
I20260619 08:18:17.070266 2527 external_mini_cluster.cc:1428] Setting key a456faf5e361588d18d0a9f96cd1568d
I20260619 08:18:17.071262 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.119.195:0
--local_ip_for_outbound_sockets=127.2.119.195
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--builtin_ntp_servers=127.2.119.212:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 08:18:17.153692 22532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:17.153894 22532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:17.153918 22532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:17.154989 22532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:17.155022 22532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:17.155046 22532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.119.195
I20260619 08:18:17.156436 22532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.119.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.119.195
--webserver_port=0
--tserver_master_addrs=127.2.119.254:42945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.119.195
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:17.156623 22532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:17.156863 22532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:17.161759 22538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:17.161767 22540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:17.161763 22537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:17.161798 22532 server_base.cc:1034] running on GCE node
I20260619 08:18:17.162231 22532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:17.162487 22532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:17.163674 22532 hybrid_clock.cc:648] HybridClock initialized: now 1781857097163664 us; error 35 us; skew 500 ppm
I20260619 08:18:17.164963 22532 webserver.cc:460] Webserver started at http://127.2.119.195:41111/ using document root <none> and password file <none>
I20260619 08:18:17.165226 22532 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:17.165356 22532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:17.165519 22532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:17.166568 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data/instance:
uuid: "ede6ef2ad84144e7a9bdb0f84b157961"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "3c013a58f234c49aca84e58ab2306121"
server_key_iv: "c1e4b53c0ed356d92ab6b70d8c013255"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.166927 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance:
uuid: "ede6ef2ad84144e7a9bdb0f84b157961"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "3c013a58f234c49aca84e58ab2306121"
server_key_iv: "c1e4b53c0ed356d92ab6b70d8c013255"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.168256 22532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:17.169083 22546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.169255 22532 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 08:18:17.169389 22532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal
uuid: "ede6ef2ad84144e7a9bdb0f84b157961"
format_stamp: "Formatted at 2026-06-19 08:18:17 on dist-test-slave-d0b5"
server_key: "3c013a58f234c49aca84e58ab2306121"
server_key_iv: "c1e4b53c0ed356d92ab6b70d8c013255"
server_key_version: "encryptionkey@0"
I20260619 08:18:17.169484 22532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:17.194175 22532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:17.194518 22532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:17.194679 22532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:17.194942 22532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 08:18:17.195284 22532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 08:18:17.195346 22532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.195394 22532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 08:18:17.195430 22532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:17.202402 22532 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.195:44401
I20260619 08:18:17.202450 22659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.195:44401 every 8 connection(s)
I20260619 08:18:17.202813 22532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/data/info.pb
I20260619 08:18:17.206570 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22532
I20260619 08:18:17.206697 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal/instance
I20260619 08:18:17.206921 2527 external_mini_cluster.cc:1428] Setting key 162b1072d81eeeb0e0aecfa0981a4b0b
I20260619 08:18:17.209467 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.203597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.195:50325 (local address 127.2.119.254:42945)
0619 08:18:17.203677 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.203679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.203932 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:18:17.204727 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.204728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.204741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.204813 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:17.205783 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.206277 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.207962 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.208147 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.208202 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.209191 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:17.209192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:17.209193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:17.209197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:17.209210 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:17.209214 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:17.209258 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.209377 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.209401 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 08:18:17.209774 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.203572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:42945 (local address 127.2.119.195:50325)
0619 08:18:17.203763 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.203813 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.204604 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.204892 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.204897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.204910 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:17.205679 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.205687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.206376 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.206378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.207880 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.207883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.208232 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.208233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.208406 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.209032 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 08:18:17.209032 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:17.209074 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 08:18:17.209102 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:17.209260 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:17.209264 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:17.209267 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.209382 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.209501 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260619 08:18:17.210285 22660 heartbeater.cc:347] Connected to a master server at 127.2.119.254:42945
I20260619 08:18:17.210389 22660 heartbeater.cc:464] Registering TS with master...
I20260619 08:18:17.210568 22660 heartbeater.cc:511] Master 127.2.119.254:42945 requested a full tablet report, sending...
I20260619 08:18:17.210992 22210 ts_manager.cc:194] Registered new tserver with Master: ede6ef2ad84144e7a9bdb0f84b157961 (127.2.119.195:44401)
I20260619 08:18:17.211349 22210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.119.195:50325
I20260619 08:18:17.218674 2527 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 08:18:17.223592 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.220154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.2.119.254:42945)
0619 08:18:17.220201 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.220203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.220306 (+ 103us) server_negotiation.cc:407] Connection header received
0619 08:18:17.220387 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.220388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.220397 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.220449 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:17.220620 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.221156 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.222442 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.222620 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.222673 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.223354 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:17.223355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:17.223355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:17.223359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:17.223371 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:17.223374 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:17.223416 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.223496 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.223517 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 08:18:17.225203 22210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 08:18:17.227005 22210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 08:18:17.234310 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.194:33635 (local address 127.0.0.1:50946)
0619 08:18:17.229415 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.229455 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.229517 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.229758 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.229761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.229763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:17.229825 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.229826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.231488 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.231489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.232564 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.232566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.233936 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.233938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.234042 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.234058 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.234131 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.234197 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":65,"threads_started":1}
I20260619 08:18:17.234669 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.2.119.194:33635)
0619 08:18:17.229600 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.229606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.229616 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:17.229657 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.229659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.229675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.229755 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:17.230064 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.231396 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.232644 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.233590 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.233670 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.233894 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.234498 (+ 604us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.234565 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260619 08:18:17.235550 22463 tablet_service.cc:1467] Processing CreateTablet for tablet 137c3bc450ec481ba6c50a8d347974fb (DEFAULT_TABLE table=test-table [id=78d106af46df434daccc4c4e71a515aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:17.235967 22463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 137c3bc450ec481ba6c50a8d347974fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:17.238114 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44401 (local address 127.0.0.1:53478)
0619 08:18:17.229843 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.229852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.229893 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.233607 (+ 3714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.233608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.233610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:17.233661 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.233663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.235007 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.235008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.236523 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.236526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.237815 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.237816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.237893 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.237897 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.237951 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.238001 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":6,"thread_start_us":56,"threads_started":1}
I20260619 08:18:17.238456 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.2.119.195:44401)
0619 08:18:17.233459 (+ 4065us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.233463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.233472 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:17.233503 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.233505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.233517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.233610 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:17.233747 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.234910 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.236632 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.237586 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.237666 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.237764 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.238299 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.238349 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3998,"thread_start_us":76,"threads_started":1}
I20260619 08:18:17.238996 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.2.119.193:34239)
0619 08:18:17.233747 (+ 4354us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.233752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.233763 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:17.233802 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.233804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.233818 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.233888 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:17.234094 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.236079 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.237722 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.238629 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.238705 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.238791 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.238882 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.238918 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4307,"thread_start_us":80,"threads_started":1}
I20260619 08:18:17.239019 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.229140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34239 (local address 127.0.0.1:36578)
0619 08:18:17.229607 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.229615 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.229652 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.233978 (+ 4326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.233979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.233981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:17.234017 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.234018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.236170 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.236171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.237629 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.237632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.238704 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.238705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.238795 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.238799 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.238869 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.238919 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":83,"threads_started":1}
I20260619 08:18:17.239989 22332 tablet_service.cc:1467] Processing CreateTablet for tablet 137c3bc450ec481ba6c50a8d347974fb (DEFAULT_TABLE table=test-table [id=78d106af46df434daccc4c4e71a515aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:17.240371 22332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 137c3bc450ec481ba6c50a8d347974fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:17.240935 22594 tablet_service.cc:1467] Processing CreateTablet for tablet 137c3bc450ec481ba6c50a8d347974fb (DEFAULT_TABLE table=test-table [id=78d106af46df434daccc4c4e71a515aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 08:18:17.241220 22594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 137c3bc450ec481ba6c50a8d347974fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:17.243597 22675 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Bootstrap starting.
I20260619 08:18:17.243841 22674 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Bootstrap starting.
I20260619 08:18:17.244477 22674 tablet_bootstrap.cc:654] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:17.244477 22675 tablet_bootstrap.cc:654] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:17.244817 22674 log.cc:824] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:17.245041 22675 log.cc:824] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:17.245427 22676 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: Bootstrap starting.
I20260619 08:18:17.245653 22674 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: No bootstrap required, opened a new log
I20260619 08:18:17.245731 22674 ts_tablet_manager.cc:1397] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 08:18:17.246125 22676 tablet_bootstrap.cc:654] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:17.246524 22676 log.cc:824] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:17.247377 22676 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: No bootstrap required, opened a new log
I20260619 08:18:17.247457 22676 ts_tablet_manager.cc:1397] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:17.248610 22674 raft_consensus.cc:348] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.248777 22674 raft_consensus.cc:374] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:17.248826 22674 raft_consensus.cc:729] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24724fb5239a456e9592bee61d4bcaf1, State: Initialized, Role: FOLLOWER
I20260619 08:18:17.248961 22674 consensus_queue.cc:260] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.249207 22674 ts_tablet_manager.cc:1428] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:17.249373 22529 heartbeater.cc:503] Master 127.2.119.254:42945 was elected leader, sending a full tablet report...
I20260619 08:18:17.250051 22675 tablet_bootstrap.cc:492] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: No bootstrap required, opened a new log
I20260619 08:18:17.250141 22675 ts_tablet_manager.cc:1397] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260619 08:18:17.250118 22676 raft_consensus.cc:348] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.250313 22676 raft_consensus.cc:374] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:17.250358 22676 raft_consensus.cc:729] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181943fb328547e4961ff60f92173df2, State: Initialized, Role: FOLLOWER
I20260619 08:18:17.250483 22676 consensus_queue.cc:260] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.250727 22676 ts_tablet_manager.cc:1428] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 08:18:17.250856 22398 heartbeater.cc:503] Master 127.2.119.254:42945 was elected leader, sending a full tablet report...
I20260619 08:18:17.252568 22675 raft_consensus.cc:348] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.252681 22675 raft_consensus.cc:374] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:17.252712 22675 raft_consensus.cc:729] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede6ef2ad84144e7a9bdb0f84b157961, State: Initialized, Role: FOLLOWER
I20260619 08:18:17.252831 22675 consensus_queue.cc:260] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.253070 22675 ts_tablet_manager.cc:1428] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 08:18:17.253155 22660 heartbeater.cc:503] Master 127.2.119.254:42945 was elected leader, sending a full tablet report...
W20260619 08:18:17.311844 22530 tablet.cc:2369] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 08:18:17.333384 22680 raft_consensus.cc:482] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 08:18:17.333508 22680 raft_consensus.cc:504] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.333837 22680 leader_election.cc:288] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 181943fb328547e4961ff60f92173df2 (127.2.119.193:34239), ede6ef2ad84144e7a9bdb0f84b157961 (127.2.119.195:44401)
I20260619 08:18:17.337852 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.334146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34239 (local address 127.2.119.194:42851)
0619 08:18:17.334332 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.334341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.334395 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.334559 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.334559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.334561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:17.334605 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.334606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.335472 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.335473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.336742 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.336745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.337546 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.337547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.337637 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.337643 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.337708 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.337751 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":13,"thread_start_us":100,"threads_started":1}
I20260619 08:18:17.337878 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.334194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:42851 (local address 127.2.119.193:34239)
0619 08:18:17.334238 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.334240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.334421 (+ 181us) server_negotiation.cc:407] Connection header received
0619 08:18:17.334457 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.334458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.334466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.334543 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:17.334684 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.335402 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.336856 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.337452 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.337528 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.337536 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.337758 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.337808 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:17.337946 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.334118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.119.194:59981 (local address 127.2.119.195:44401)
0619 08:18:17.334211 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.334213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.334246 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:18:17.334279 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.334280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.334290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.334348 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 08:18:17.334515 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.335334 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.336880 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.337452 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.337528 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.337536 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.337762 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.337805 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 08:18:17.338114 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.334058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.195:44401 (local address 127.2.119.194:59981)
0619 08:18:17.334131 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.334143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.334189 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.334360 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:17.334361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:17.334363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 08:18:17.334438 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.334440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.335419 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.335421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.336742 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:17.336745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.337546 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.337547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:17.337637 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.337643 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 08:18:17.337708 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 08:18:17.337752 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":76}
I20260619 08:18:17.338300 22352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "137c3bc450ec481ba6c50a8d347974fb" candidate_uuid: "24724fb5239a456e9592bee61d4bcaf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181943fb328547e4961ff60f92173df2" is_pre_election: true
I20260619 08:18:17.338439 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "137c3bc450ec481ba6c50a8d347974fb" candidate_uuid: "24724fb5239a456e9592bee61d4bcaf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" is_pre_election: true
I20260619 08:18:17.338483 22352 raft_consensus.cc:2454] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24724fb5239a456e9592bee61d4bcaf1 in term 0.
I20260619 08:18:17.338564 22614 raft_consensus.cc:2454] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24724fb5239a456e9592bee61d4bcaf1 in term 0.
I20260619 08:18:17.338740 22419 leader_election.cc:302] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24724fb5239a456e9592bee61d4bcaf1, ede6ef2ad84144e7a9bdb0f84b157961; no voters:
I20260619 08:18:17.338922 22680 raft_consensus.cc:2781] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 08:18:17.339025 22680 raft_consensus.cc:482] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 08:18:17.339062 22680 raft_consensus.cc:3037] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:17.339816 22680 raft_consensus.cc:504] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.339978 22680 leader_election.cc:288] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [CANDIDATE]: Term 1 election: Requested vote from peers 181943fb328547e4961ff60f92173df2 (127.2.119.193:34239), ede6ef2ad84144e7a9bdb0f84b157961 (127.2.119.195:44401)
I20260619 08:18:17.340116 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "137c3bc450ec481ba6c50a8d347974fb" candidate_uuid: "24724fb5239a456e9592bee61d4bcaf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede6ef2ad84144e7a9bdb0f84b157961"
I20260619 08:18:17.340169 22352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "137c3bc450ec481ba6c50a8d347974fb" candidate_uuid: "24724fb5239a456e9592bee61d4bcaf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181943fb328547e4961ff60f92173df2"
I20260619 08:18:17.340198 22614 raft_consensus.cc:3037] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:17.340235 22352 raft_consensus.cc:3037] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:17.341146 22614 raft_consensus.cc:2454] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24724fb5239a456e9592bee61d4bcaf1 in term 1.
I20260619 08:18:17.341146 22352 raft_consensus.cc:2454] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24724fb5239a456e9592bee61d4bcaf1 in term 1.
I20260619 08:18:17.341382 22419 leader_election.cc:302] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24724fb5239a456e9592bee61d4bcaf1, ede6ef2ad84144e7a9bdb0f84b157961; no voters:
I20260619 08:18:17.341487 22680 raft_consensus.cc:2781] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:17.341629 22680 raft_consensus.cc:686] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 1 LEADER]: Becoming Leader. State: Replica: 24724fb5239a456e9592bee61d4bcaf1, State: Running, Role: LEADER
I20260619 08:18:17.341753 22680 consensus_queue.cc:237] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } }
I20260619 08:18:17.342638 22209 catalog_manager.cc:5495] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 24724fb5239a456e9592bee61d4bcaf1 (127.2.119.194). New cstate: current_term: 1 leader_uuid: "24724fb5239a456e9592bee61d4bcaf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24724fb5239a456e9592bee61d4bcaf1" member_type: VOTER last_known_addr { host: "127.2.119.194" port: 33635 } health_report { overall_health: HEALTHY } } }
I20260619 08:18:17.349050 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:17.346686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.2.119.194:33635)
0619 08:18:17.346792 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:17.346794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:17.346803 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:17.346832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:17.346833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:17.346851 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:17.346909 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:17.347029 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.347486 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.348281 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:17.348470 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:17.348541 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:17.348673 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:17.348833 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:17.348938 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:17.348954 (+ 16us) server_negotiation.cc:299] Negotiation successful
0619 08:18:17.348974 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:17.350719 22352 raft_consensus.cc:1261] T 137c3bc450ec481ba6c50a8d347974fb P 181943fb328547e4961ff60f92173df2 [term 1 FOLLOWER]: Refusing update from remote peer 24724fb5239a456e9592bee61d4bcaf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:17.350746 22614 raft_consensus.cc:1261] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 1 FOLLOWER]: Refusing update from remote peer 24724fb5239a456e9592bee61d4bcaf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:17.351116 22684 consensus_queue.cc:1035] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "181943fb328547e4961ff60f92173df2" member_type: VOTER last_known_addr { host: "127.2.119.193" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:17.351226 22680 consensus_queue.cc:1035] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede6ef2ad84144e7a9bdb0f84b157961" member_type: VOTER last_known_addr { host: "127.2.119.195" port: 44401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:17.354461 22689 mvcc.cc:205] Tried to move back new op lower bound from 7298486670746193920 to 7298486670712614912. Current Snapshot: MvccSnapshot[applied={T|T < 7298486670746193920}]
I20260619 08:18:17.355682 22209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:17.355796 22209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 08:18:17.356595 22209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3c2340713f04605ad011a8d23500a44: Sending DeleteTablet for 3 replicas of tablet 137c3bc450ec481ba6c50a8d347974fb
I20260619 08:18:17.357005 22332 tablet_service.cc:1514] Processing DeleteTablet for tablet 137c3bc450ec481ba6c50a8d347974fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:17 UTC) from {username='slave'} at 127.0.0.1:36578
I20260619 08:18:17.357214 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22270
I20260619 08:18:17.357393 22594 tablet_service.cc:1514] Processing DeleteTablet for tablet 137c3bc450ec481ba6c50a8d347974fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:17 UTC) from {username='slave'} at 127.0.0.1:53478
I20260619 08:18:17.357231 22463 tablet_service.cc:1514] Processing DeleteTablet for tablet 137c3bc450ec481ba6c50a8d347974fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 08:18:17 UTC) from {username='slave'} at 127.0.0.1:50946
I20260619 08:18:17.358045 22700 tablet_replica.cc:330] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: stopping tablet replica
I20260619 08:18:17.358136 22700 raft_consensus.cc:2229] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 08:18:17.358223 22700 raft_consensus.cc:2258] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:17.358693 22700 ts_tablet_manager.cc:1905] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 08:18:17.360674 22700 ts_tablet_manager.cc:1918] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:17.360739 22700 log.cc:1196] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-1/wal/wals/137c3bc450ec481ba6c50a8d347974fb
I20260619 08:18:17.361034 22700 ts_tablet_manager.cc:1939] T 137c3bc450ec481ba6c50a8d347974fb P 24724fb5239a456e9592bee61d4bcaf1: Deleting consensus metadata
I20260619 08:18:17.361721 22196 catalog_manager.cc:4841] TS 24724fb5239a456e9592bee61d4bcaf1 (127.2.119.194:33635): tablet 137c3bc450ec481ba6c50a8d347974fb (table test-table [id=78d106af46df434daccc4c4e71a515aa]) successfully deleted
I20260619 08:18:17.361757 22699 tablet_replica.cc:330] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: stopping tablet replica
I20260619 08:18:17.361955 22699 raft_consensus.cc:2229] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 08:18:17.362076 22699 raft_consensus.cc:2258] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 08:18:17.362484 22699 ts_tablet_manager.cc:1905] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 08:18:17.363894 22194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.119.193:34239 (error 108)
I20260619 08:18:17.364050 22699 ts_tablet_manager.cc:1918] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 08:18:17.364131 22699 log.cc:1196] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Deleting WAL directory at /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781856856097588-2527-0/minicluster-data/ts-2/wal/wals/137c3bc450ec481ba6c50a8d347974fb
I20260619 08:18:17.364382 22699 ts_tablet_manager.cc:1939] T 137c3bc450ec481ba6c50a8d347974fb P ede6ef2ad84144e7a9bdb0f84b157961: Deleting consensus metadata
W20260619 08:18:17.364413 22669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:17.364217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.193:34239 (local address 127.0.0.1:36594)
0619 08:18:17.364257 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:17.364269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:17.364318 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:17.364358 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34239: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":19}
W20260619 08:18:17.364511 22194 catalog_manager.cc:4584] TS 181943fb328547e4961ff60f92173df2 (127.2.119.193:34239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 137c3bc450ec481ba6c50a8d347974fb: Network error: Client connection negotiation failed: client connection to 127.2.119.193:34239: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260619 08:18:17.364815 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22401
I20260619 08:18:17.366886 22197 catalog_manager.cc:4841] TS ede6ef2ad84144e7a9bdb0f84b157961 (127.2.119.195:44401): tablet 137c3bc450ec481ba6c50a8d347974fb (table test-table [id=78d106af46df434daccc4c4e71a515aa]) successfully deleted
I20260619 08:18:17.370092 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22532
I20260619 08:18:17.374660 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22179
2026-06-19T08:18:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (780 ms)
[----------] 6 tests from EncryptionPolicyTest (8981 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:17 dist-test-slave-d0b5 krb5kdc[22704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:17 dist-test-slave-d0b5 krb5kdc[22704](info): set up 2 sockets
Jun 19 08:18:17 dist-test-slave-d0b5 krb5kdc[22704](info): commencing operation
krb5kdc: starting...
W20260619 08:18:18.422493 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:18.540807 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:18.639536 22717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:18.639822 22717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:18.639859 22717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:18.641705 22717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:18.641758 22717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:18.641781 22717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:18.641803 22717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:18.641827 22717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:18.644060 22717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:18.644320 22717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:18.644556 22717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:18.649816 22722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:18.649852 22717 server_base.cc:1034] running on GCE node
W20260619 08:18:18.649835 22725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:18.650101 22723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:18.650393 22717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:18.650693 22717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:18.651873 22717 hybrid_clock.cc:648] HybridClock initialized: now 1781857098651850 us; error 40 us; skew 500 ppm
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857098, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:18.654371 22717 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:18.654748 22717 webserver.cc:460] Webserver started at http://127.0.0.1:37921/ using document root <none> and password file <none>
I20260619 08:18:18.654927 22717 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:18.654965 22717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:18.655053 22717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:18.656162 22717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "b2c91cc748434d5b92414a8f99ce64c0"
format_stamp: "Formatted at 2026-06-19 08:18:18 on dist-test-slave-d0b5"
server_key: "5563ab6d804b7f74b5b25f3a0c9c5386"
server_key_iv: "c7b8daac4c1c2a5f53145290a93c2a16"
server_key_version: "encryptionkey@0"
I20260619 08:18:18.656472 22717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "b2c91cc748434d5b92414a8f99ce64c0"
format_stamp: "Formatted at 2026-06-19 08:18:18 on dist-test-slave-d0b5"
server_key: "5563ab6d804b7f74b5b25f3a0c9c5386"
server_key_iv: "c7b8daac4c1c2a5f53145290a93c2a16"
server_key_version: "encryptionkey@0"
I20260619 08:18:18.657824 22717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:18.658597 22732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:18.658819 22717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:18.658947 22717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "b2c91cc748434d5b92414a8f99ce64c0"
format_stamp: "Formatted at 2026-06-19 08:18:18 on dist-test-slave-d0b5"
server_key: "5563ab6d804b7f74b5b25f3a0c9c5386"
server_key_iv: "c7b8daac4c1c2a5f53145290a93c2a16"
server_key_version: "encryptionkey@0"
I20260619 08:18:18.659006 22717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:18.700609 22717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:18.702732 22717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:18.702941 22717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:18.707166 22717 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36191
I20260619 08:18:18.707233 22784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36191 every 8 connection(s)
I20260619 08:18:18.707589 22717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:18.708628 22785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:18.710943 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: Bootstrap starting.
I20260619 08:18:18.711544 22785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:18.711930 22785 log.cc:824] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:18.712662 22785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: No bootstrap required, opened a new log
I20260619 08:18:18.714347 22785 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260619 08:18:18.714485 22785 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:18.714509 22785 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c91cc748434d5b92414a8f99ce64c0, State: Initialized, Role: FOLLOWER
I20260619 08:18:18.714672 22785 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260619 08:18:18.714744 22785 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:18.714804 22785 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:18.714856 22785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:18.715402 22785 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260619 08:18:18.715485 22785 leader_election.cc:302] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c91cc748434d5b92414a8f99ce64c0; no voters:
I20260619 08:18:18.715670 22785 leader_election.cc:288] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:18.715749 22788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:18.715958 22785 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:18.716046 22788 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [term 1 LEADER]: Becoming Leader. State: Replica: b2c91cc748434d5b92414a8f99ce64c0, State: Running, Role: LEADER
I20260619 08:18:18.716178 22788 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } }
I20260619 08:18:18.716401 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22717
I20260619 08:18:18.716506 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:18.716701 2527 external_mini_cluster.cc:1428] Setting key 7f498147aa61555e9f98751026b679ac
I20260619 08:18:18.717073 22790 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c91cc748434d5b92414a8f99ce64c0. Latest consensus state: current_term: 1 leader_uuid: "b2c91cc748434d5b92414a8f99ce64c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } } }
I20260619 08:18:18.717592 22789 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c91cc748434d5b92414a8f99ce64c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c91cc748434d5b92414a8f99ce64c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36191 } } }
I20260619 08:18:18.717801 22789 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:18.717556 22790 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0 [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:18.718457 22806 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:18.718523 22806 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:18.718571 22805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:18.718732 22805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:18.720467 22805 catalog_manager.cc:1347] Generated new cluster ID: 662af5457dac4d7e92edc193a2f560c5
I20260619 08:18:18.720521 22805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:18:18.728358 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:18.717578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45433 (local address 127.0.0.1:36191)
0619 08:18:18.717916 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:18.717925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:18.717962 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:18:18.718080 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:18.718087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:18.718144 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:18.718291 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:18.719161 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:18.720614 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:18.721495 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:18.721916 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:18.722034 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:18.724419 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:18.724423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:18.724423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:18.724436 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:18.726758 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:18.727172 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:18.727181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:18.727182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:18.727278 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:18.727517 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:18.727518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:18.727519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:18.727761 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:18.727948 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:18.728043 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 08:18:18.728150 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":87,"threads_started":1}
I20260619 08:18:18.729406 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:18.735798 22805 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:18.736537 22805 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:18.737128 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:18.730772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51743 (local address 127.0.0.1:36191)
0619 08:18:18.730811 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:18.730813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:18.730949 (+ 136us) server_negotiation.cc:407] Connection header received
0619 08:18:18.731017 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:18.731018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:18.731027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:18.731071 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:18.731585 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:18.732079 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:18.732771 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:18.733007 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:18.733092 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:18.734284 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:18.734285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:18.734286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:18.734293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:18.735770 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:18.736228 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:18.736229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:18.736229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:18.736280 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:18.736523 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:18.736523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:18.736524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:18.736637 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:18.736714 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:18.736969 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 08:18:18.737037 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:18.750133 22805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c91cc748434d5b92414a8f99ce64c0: Generated new TSK 0
I20260619 08:18:18.750334 22805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:18.753180 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22717
W20260619 08:18:18.757066 22702 connection.cc:535] client connection to 127.0.0.1:36191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36191): Connection reset by peer (error 104)
2026-06-19T08:18:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22816](info): set up 2 sockets
Jun 19 08:18:18 dist-test-slave-d0b5 krb5kdc[22816](info): commencing operation
krb5kdc: starting...
W20260619 08:18:19.806465 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:19 dist-test-slave-d0b5 krb5kdc[22816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:19.928435 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:20.008917 22829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:20.009173 22829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:20.009214 22829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:20.010419 22829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:20.010497 22829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:20.010524 22829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:20.010557 22829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:20.010581 22829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:20.012086 22829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:20.012374 22829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:20.012605 22829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:20.017460 22837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:20.017450 22834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:20.017448 22835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:20.017740 22829 server_base.cc:1034] running on GCE node
I20260619 08:18:20.018002 22829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:20.018282 22829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:20.019454 22829 hybrid_clock.cc:648] HybridClock initialized: now 1781857100019421 us; error 45 us; skew 500 ppm
Jun 19 08:18:20 dist-test-slave-d0b5 krb5kdc[22816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857100, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:20.022087 22829 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:20.022567 22829 webserver.cc:460] Webserver started at http://127.0.0.1:44457/ using document root <none> and password file <none>
I20260619 08:18:20.022771 22829 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:20.022812 22829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:20.022900 22829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:20.023902 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "0a0f74cb4d254ae5b819fadcc8a23553"
format_stamp: "Formatted at 2026-06-19 08:18:20 on dist-test-slave-d0b5"
server_key: "77bddb2080a641c2581894f79c012fba"
server_key_iv: "8240f4af18571e42962580775c6ac44f"
server_key_version: "encryptionkey@0"
I20260619 08:18:20.024201 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "0a0f74cb4d254ae5b819fadcc8a23553"
format_stamp: "Formatted at 2026-06-19 08:18:20 on dist-test-slave-d0b5"
server_key: "77bddb2080a641c2581894f79c012fba"
server_key_iv: "8240f4af18571e42962580775c6ac44f"
server_key_version: "encryptionkey@0"
I20260619 08:18:20.025637 22829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 08:18:20.026393 22844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:20.026602 22829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:20.026693 22829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "0a0f74cb4d254ae5b819fadcc8a23553"
format_stamp: "Formatted at 2026-06-19 08:18:20 on dist-test-slave-d0b5"
server_key: "77bddb2080a641c2581894f79c012fba"
server_key_iv: "8240f4af18571e42962580775c6ac44f"
server_key_version: "encryptionkey@0"
I20260619 08:18:20.026777 22829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:20.044559 22829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:20.047104 22829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:20.047273 22829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:20.051554 22829 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37853
I20260619 08:18:20.051591 22896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37853 every 8 connection(s)
I20260619 08:18:20.051976 22829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:20.052955 22897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:20.053339 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22829
I20260619 08:18:20.053453 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:20.053646 2527 external_mini_cluster.cc:1428] Setting key 5d97f10aaa8c6be87232beddb62b0590
I20260619 08:18:20.055874 22897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553: Bootstrap starting.
I20260619 08:18:20.056540 22897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:20.056836 22897 log.cc:824] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:20.057547 22897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553: No bootstrap required, opened a new log
I20260619 08:18:20.059235 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:20.054210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53185 (local address 127.0.0.1:37853)
0619 08:18:20.054378 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:20.054389 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:20.054418 (+ 29us) server_negotiation.cc:407] Connection header received
0619 08:18:20.054811 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:20.054817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:20.054859 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:20.054980 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:20.055241 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:20.056244 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:20.057366 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:20.057827 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:20.057921 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:20.058605 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:20.058609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:20.058609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:20.058766 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:20.058796 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:20.058806 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:20.058870 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:20.058984 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:18:20.059032 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260619 08:18:20.059829 22897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } }
I20260619 08:18:20.060005 22897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:20.060055 22897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0f74cb4d254ae5b819fadcc8a23553, State: Initialized, Role: FOLLOWER
I20260619 08:18:20.060176 22897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } }
I20260619 08:18:20.060242 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:20.060261 22897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:20.060294 22897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:20.060346 22897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:20.061019 22897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } }
I20260619 08:18:20.061193 22897 leader_election.cc:302] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a0f74cb4d254ae5b819fadcc8a23553; no voters:
I20260619 08:18:20.061445 22897 leader_election.cc:288] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:20.061486 22904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:20.061633 22904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [term 1 LEADER]: Becoming Leader. State: Replica: 0a0f74cb4d254ae5b819fadcc8a23553, State: Running, Role: LEADER
I20260619 08:18:20.061715 22897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:20.061730 22904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } }
I20260619 08:18:20.062072 22904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } } }
I20260619 08:18:20.062093 22908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a0f74cb4d254ae5b819fadcc8a23553. Latest consensus state: current_term: 1 leader_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a0f74cb4d254ae5b819fadcc8a23553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37853 } } }
I20260619 08:18:20.062166 22904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:20.062175 22908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:20.062387 22913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:20.062873 22913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:20.065212 22913 catalog_manager.cc:1347] Generated new cluster ID: 3c094e1dc2b14264afcde1539ba1163b
I20260619 08:18:20.065272 22913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:20.065555 22900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:20.061854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47133 (local address 127.0.0.1:37853)
0619 08:18:20.061908 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:20.061910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:20.062116 (+ 206us) server_negotiation.cc:407] Connection header received
0619 08:18:20.062309 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:20.062310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:20.062320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:20.062383 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:20.062592 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:20.063284 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:20.064199 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:20.064499 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:20.064574 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:20.065282 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:20.065284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:20.065284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:20.065289 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:20.065334 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:20.065339 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:20.065404 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:20.065481 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:18:20.065490 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 08:18:20.082490 22913 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:20.083253 22913 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:20.107681 22913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a0f74cb4d254ae5b819fadcc8a23553: Generated new TSK 0
I20260619 08:18:20.107920 22913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:20.119282 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22829
W20260619 08:18:20.123651 22814 connection.cc:535] client connection to 127.0.0.1:37853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37853): Connection reset by peer (error 104)
2026-06-19T08:18:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:20 dist-test-slave-d0b5 krb5kdc[22928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:20 dist-test-slave-d0b5 krb5kdc[22928](info): set up 2 sockets
Jun 19 08:18:20 dist-test-slave-d0b5 krb5kdc[22928](info): commencing operation
krb5kdc: starting...
W20260619 08:18:21.166313 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:21 dist-test-slave-d0b5 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:21.285203 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:21.364856 22941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:21.365078 22941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:21.365116 22941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:21.366317 22941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:21.366381 22941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:21.366407 22941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:21.366436 22941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:21.366462 22941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:21.367885 22941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39637
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:21.368142 22941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:21.368356 22941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:21.373152 22949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:21.373174 22947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:21.373152 22946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:21.373498 22941 server_base.cc:1034] running on GCE node
I20260619 08:18:21.373806 22941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:21.374081 22941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:21.375279 22941 hybrid_clock.cc:648] HybridClock initialized: now 1781857101375255 us; error 34 us; skew 500 ppm
Jun 19 08:18:21 dist-test-slave-d0b5 krb5kdc[22928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857101, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:21.377835 22941 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:21.378257 22941 webserver.cc:460] Webserver started at http://127.0.0.1:33929/ using document root <none> and password file <none>
I20260619 08:18:21.378505 22941 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:21.378576 22941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:21.378720 22941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:21.379803 22941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "dba88b8245a848a8b0ff1ac732bf9f4e"
format_stamp: "Formatted at 2026-06-19 08:18:21 on dist-test-slave-d0b5"
server_key: "e8a91b2cda2b795533054142cf1b28ef"
server_key_iv: "aa3dc543733595f4c494893562264610"
server_key_version: "encryptionkey@0"
I20260619 08:18:21.380164 22941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "dba88b8245a848a8b0ff1ac732bf9f4e"
format_stamp: "Formatted at 2026-06-19 08:18:21 on dist-test-slave-d0b5"
server_key: "e8a91b2cda2b795533054142cf1b28ef"
server_key_iv: "aa3dc543733595f4c494893562264610"
server_key_version: "encryptionkey@0"
I20260619 08:18:21.381507 22941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:18:21.382216 22956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:21.382478 22941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:21.382611 22941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "dba88b8245a848a8b0ff1ac732bf9f4e"
format_stamp: "Formatted at 2026-06-19 08:18:21 on dist-test-slave-d0b5"
server_key: "e8a91b2cda2b795533054142cf1b28ef"
server_key_iv: "aa3dc543733595f4c494893562264610"
server_key_version: "encryptionkey@0"
I20260619 08:18:21.382695 22941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:21.385989 22941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:21.388175 22941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:21.388329 22941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:21.392232 22941 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39637
I20260619 08:18:21.392273 23008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39637 every 8 connection(s)
I20260619 08:18:21.392611 22941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:21.393627 23009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:21.395843 23009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e: Bootstrap starting.
I20260619 08:18:21.396358 23009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:21.396581 23009 log.cc:824] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:21.397220 23009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e: No bootstrap required, opened a new log
I20260619 08:18:21.398763 23009 raft_consensus.cc:348] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260619 08:18:21.398895 23009 raft_consensus.cc:374] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:21.398916 23009 raft_consensus.cc:729] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dba88b8245a848a8b0ff1ac732bf9f4e, State: Initialized, Role: FOLLOWER
I20260619 08:18:21.399045 23009 consensus_queue.cc:260] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260619 08:18:21.399122 23009 raft_consensus.cc:388] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:21.399145 23009 raft_consensus.cc:482] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:21.399173 23009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:21.399658 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 22941
I20260619 08:18:21.399780 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:21.399756 23009 raft_consensus.cc:504] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260619 08:18:21.399852 23009 leader_election.cc:302] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dba88b8245a848a8b0ff1ac732bf9f4e; no voters:
I20260619 08:18:21.399968 2527 external_mini_cluster.cc:1428] Setting key c2833106f001537f192f6b68e53102c5
I20260619 08:18:21.400014 23009 leader_election.cc:288] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:21.400080 23012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:21.400235 23012 raft_consensus.cc:686] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [term 1 LEADER]: Becoming Leader. State: Replica: dba88b8245a848a8b0ff1ac732bf9f4e, State: Running, Role: LEADER
I20260619 08:18:21.400257 23009 sys_catalog.cc:564] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:21.400319 23012 consensus_queue.cc:237] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260619 08:18:21.400628 23013 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260619 08:18:21.400727 23013 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:21.400947 23018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:21.400938 23014 sys_catalog.cc:455] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dba88b8245a848a8b0ff1ac732bf9f4e. Latest consensus state: current_term: 1 leader_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dba88b8245a848a8b0ff1ac732bf9f4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260619 08:18:21.401079 23014 sys_catalog.cc:458] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:21.401675 23018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:21.402797 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:21.400438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50545 (local address 127.0.0.1:39637)
0619 08:18:21.400982 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:21.400993 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:21.401030 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:18:21.401156 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:21.401165 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:21.401197 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:21.402119 (+ 922us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:21.402359 (+ 240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:21.402361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:21.402361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:21.402364 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:21.402389 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:21.402398 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:21.402489 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:21.402585 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 08:18:21.402628 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":83,"threads_started":1}
I20260619 08:18:21.403574 23018 catalog_manager.cc:1347] Generated new cluster ID: 7208a85751204e73ae44628e1410bd75
I20260619 08:18:21.403625 23018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:21.403884 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:21.406725 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:21.405615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43047 (local address 127.0.0.1:39637)
0619 08:18:21.405677 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:21.405679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:21.405859 (+ 180us) server_negotiation.cc:407] Connection header received
0619 08:18:21.405918 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:21.405919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:21.405923 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:21.405975 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:21.406529 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:21.406530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:21.406530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:21.406534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:21.406548 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:21.406552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:21.406602 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:21.406654 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 08:18:21.406666 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:18:21.444168 23018 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:21.444396 23018 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:21.449271 23018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dba88b8245a848a8b0ff1ac732bf9f4e: Generated new TSK 0
I20260619 08:18:21.449512 23018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:21.453398 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 22941
2026-06-19T08:18:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:21 dist-test-slave-d0b5 krb5kdc[23040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:21 dist-test-slave-d0b5 krb5kdc[23040](info): set up 2 sockets
Jun 19 08:18:21 dist-test-slave-d0b5 krb5kdc[23040](info): commencing operation
krb5kdc: starting...
W20260619 08:18:22.538473 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:22.662789 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:22.742815 23053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:22.743012 23053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:22.743036 23053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:22.744122 23053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:22.744156 23053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:22.744170 23053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:22.744184 23053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:22.744195 23053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:22.745671 23053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:22.745867 23053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:22.746030 23053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:22.750885 23059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:22.750939 23058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:22.751108 23053 server_base.cc:1034] running on GCE node
W20260619 08:18:22.750947 23061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:22.751482 23053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:22.751812 23053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:22.753003 23053 hybrid_clock.cc:648] HybridClock initialized: now 1781857102752987 us; error 48 us; skew 500 ppm
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:22.755894 23053 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:22.756362 23053 webserver.cc:460] Webserver started at http://127.0.0.1:34727/ using document root <none> and password file <none>
I20260619 08:18:22.756629 23053 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:22.756706 23053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:22.756876 23053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:22.758073 23053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "d25bedea2f1647f6ad2e29b8c1f02088"
format_stamp: "Formatted at 2026-06-19 08:18:22 on dist-test-slave-d0b5"
server_key: "b92834ef0be8cc00022b29622db6c58e"
server_key_iv: "7cf5671f674431769d00baefd2dde0e4"
server_key_version: "encryptionkey@0"
I20260619 08:18:22.758462 23053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "d25bedea2f1647f6ad2e29b8c1f02088"
format_stamp: "Formatted at 2026-06-19 08:18:22 on dist-test-slave-d0b5"
server_key: "b92834ef0be8cc00022b29622db6c58e"
server_key_iv: "7cf5671f674431769d00baefd2dde0e4"
server_key_version: "encryptionkey@0"
I20260619 08:18:22.759974 23053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:18:22.760965 23068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:22.761185 23053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:22.761278 23053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "d25bedea2f1647f6ad2e29b8c1f02088"
format_stamp: "Formatted at 2026-06-19 08:18:22 on dist-test-slave-d0b5"
server_key: "b92834ef0be8cc00022b29622db6c58e"
server_key_iv: "7cf5671f674431769d00baefd2dde0e4"
server_key_version: "encryptionkey@0"
I20260619 08:18:22.761417 23053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:22.767726 23053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:22.769613 23053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:22.769765 23053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:22.773469 23053 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36771
I20260619 08:18:22.773522 23120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36771 every 8 connection(s)
I20260619 08:18:22.773862 23053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:22.774822 23121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:22.777045 23121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088: Bootstrap starting.
I20260619 08:18:22.777472 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23053
I20260619 08:18:22.777599 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:22.777776 2527 external_mini_cluster.cc:1428] Setting key 93021ec521c2e62a28010348079cefa4
I20260619 08:18:22.777791 23121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:22.778076 23121 log.cc:824] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:22.778883 23121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088: No bootstrap required, opened a new log
I20260619 08:18:22.781877 23121 raft_consensus.cc:348] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } }
I20260619 08:18:22.782101 23121 raft_consensus.cc:374] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:22.782159 23121 raft_consensus.cc:729] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25bedea2f1647f6ad2e29b8c1f02088, State: Initialized, Role: FOLLOWER
I20260619 08:18:22.782361 23121 consensus_queue.cc:260] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } }
I20260619 08:18:22.782459 23121 raft_consensus.cc:388] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:22.782493 23121 raft_consensus.cc:482] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:22.782586 23121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:22.783245 23121 raft_consensus.cc:504] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } }
I20260619 08:18:22.783339 23121 leader_election.cc:302] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d25bedea2f1647f6ad2e29b8c1f02088; no voters:
I20260619 08:18:22.783533 23121 leader_election.cc:288] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:22.783560 23126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:22.783690 23126 raft_consensus.cc:686] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [term 1 LEADER]: Becoming Leader. State: Replica: d25bedea2f1647f6ad2e29b8c1f02088, State: Running, Role: LEADER
I20260619 08:18:22.783813 23121 sys_catalog.cc:564] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:22.783843 23126 consensus_queue.cc:237] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } }
I20260619 08:18:22.784215 23126 sys_catalog.cc:455] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d25bedea2f1647f6ad2e29b8c1f02088. Latest consensus state: current_term: 1 leader_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } } }
I20260619 08:18:22.784310 23126 sys_catalog.cc:458] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:22.784416 23127 sys_catalog.cc:455] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d25bedea2f1647f6ad2e29b8c1f02088" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36771 } } }
I20260619 08:18:22.784515 23127 sys_catalog.cc:458] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:22.785200 23132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:22.785858 23132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:18:22.787938 23132 catalog_manager.cc:1347] Generated new cluster ID: 7dc9123f21ed451fa4c9305d28fe22a8
I20260619 08:18:22.787990 23132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:22.791611 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:22.778271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58595 (local address 127.0.0.1:36771)
0619 08:18:22.778433 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:22.778443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:22.778480 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:18:22.779257 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:22.779266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:22.779332 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:22.779485 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:22.780417 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:22.781948 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:22.783045 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:22.783670 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:22.784110 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:22.787394 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:22.787398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:22.787399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:22.787411 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:22.789646 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:22.790070 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:22.790073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:22.790073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:22.790171 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:22.790375 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:22.790376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:22.790376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:22.790674 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:22.790862 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:22.791275 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 08:18:22.791412 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260619 08:18:22.796303 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:22.801226 23132 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:22.801916 23132 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:22.806442 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:22.798687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34529 (local address 127.0.0.1:36771)
0619 08:18:22.798736 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:22.798738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:22.798951 (+ 213us) server_negotiation.cc:407] Connection header received
0619 08:18:22.798999 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:22.799000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:22.799011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:22.799065 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:22.799744 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:22.800567 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:22.801634 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:22.801995 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:22.802071 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:22.803571 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:22.803573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:22.803573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:22.803580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:22.805200 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:22.805623 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:22.805624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:22.805625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:22.805675 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:22.805884 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:22.805885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:22.805885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:22.805997 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:22.806065 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:22.806261 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 08:18:22.806347 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 08:18:22.813078 23132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d25bedea2f1647f6ad2e29b8c1f02088: Generated new TSK 0
I20260619 08:18:22.813339 23132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:22.817119 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23053
2026-06-19T08:18:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23152](info): set up 2 sockets
Jun 19 08:18:22 dist-test-slave-d0b5 krb5kdc[23152](info): commencing operation
krb5kdc: starting...
W20260619 08:18:23.858292 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:23 dist-test-slave-d0b5 krb5kdc[23152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:23.984637 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:24.069597 23165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:24.069854 23165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:24.069895 23165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:24.071094 23165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:24.071166 23165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:24.071192 23165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:24.071223 23165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:24.071252 23165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:24.072721 23165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:24.073004 23165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:24.073236 23165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:24.078224 23171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:24.078284 23170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:24.078226 23173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:24.079105 23165 server_base.cc:1034] running on GCE node
I20260619 08:18:24.079386 23165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:24.079651 23165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:24.080823 23165 hybrid_clock.cc:648] HybridClock initialized: now 1781857104080817 us; error 39 us; skew 500 ppm
Jun 19 08:18:24 dist-test-slave-d0b5 krb5kdc[23152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:24.087780 23165 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:24.088305 23165 webserver.cc:460] Webserver started at http://127.0.0.1:43129/ using document root <none> and password file <none>
I20260619 08:18:24.088531 23165 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:24.088573 23165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:24.088665 23165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:24.089830 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "34c278d802344691b5f2641133e50b93"
format_stamp: "Formatted at 2026-06-19 08:18:24 on dist-test-slave-d0b5"
server_key: "73eaba43ab1bd0b18a4eeb309f5febb8"
server_key_iv: "b62ea12268033eb388536c442eddff7a"
server_key_version: "encryptionkey@0"
I20260619 08:18:24.090133 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "34c278d802344691b5f2641133e50b93"
format_stamp: "Formatted at 2026-06-19 08:18:24 on dist-test-slave-d0b5"
server_key: "73eaba43ab1bd0b18a4eeb309f5febb8"
server_key_iv: "b62ea12268033eb388536c442eddff7a"
server_key_version: "encryptionkey@0"
I20260619 08:18:24.091423 23165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 08:18:24.092123 23180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:24.092343 23165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:24.092430 23165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "34c278d802344691b5f2641133e50b93"
format_stamp: "Formatted at 2026-06-19 08:18:24 on dist-test-slave-d0b5"
server_key: "73eaba43ab1bd0b18a4eeb309f5febb8"
server_key_iv: "b62ea12268033eb388536c442eddff7a"
server_key_version: "encryptionkey@0"
I20260619 08:18:24.092486 23165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:24.134244 23165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:24.136842 23165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:24.136957 23165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:24.141009 23165 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40233
I20260619 08:18:24.141093 23232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40233 every 8 connection(s)
I20260619 08:18:24.141479 23165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:24.142575 23233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:24.144830 23233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93: Bootstrap starting.
I20260619 08:18:24.145429 23233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:24.145718 23233 log.cc:824] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:24.146471 23233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93: No bootstrap required, opened a new log
I20260619 08:18:24.148043 23233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260619 08:18:24.148204 23233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:24.148248 23233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c278d802344691b5f2641133e50b93, State: Initialized, Role: FOLLOWER
I20260619 08:18:24.148452 23233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260619 08:18:24.148555 23233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:24.148602 23233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:24.148653 23233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:24.149168 23233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260619 08:18:24.149284 23233 leader_election.cc:302] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c278d802344691b5f2641133e50b93; no voters:
I20260619 08:18:24.149500 23233 leader_election.cc:288] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:24.149557 23236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:24.149742 23236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [term 1 LEADER]: Becoming Leader. State: Replica: 34c278d802344691b5f2641133e50b93, State: Running, Role: LEADER
I20260619 08:18:24.149755 23233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:24.149868 23236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } }
I20260619 08:18:24.150225 23237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c278d802344691b5f2641133e50b93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } } }
I20260619 08:18:24.150357 23237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:24.150470 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23165
I20260619 08:18:24.150591 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:24.150665 23240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:24.150779 2527 external_mini_cluster.cc:1428] Setting key 59c090698131fa9ba064c11ab575c192
I20260619 08:18:24.151199 23240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:24.150234 23238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c278d802344691b5f2641133e50b93. Latest consensus state: current_term: 1 leader_uuid: "34c278d802344691b5f2641133e50b93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c278d802344691b5f2641133e50b93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40233 } } }
I20260619 08:18:24.151551 23238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:24.153761 23240 catalog_manager.cc:1347] Generated new cluster ID: 43e5c3b3088e49cb914a8f7ecf950267
I20260619 08:18:24.153815 23240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:24.157390 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:24.151248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50899 (local address 127.0.0.1:40233)
0619 08:18:24.151508 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:24.151518 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:24.151877 (+ 359us) server_negotiation.cc:407] Connection header received
0619 08:18:24.152113 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:24.152121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:24.152199 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:24.152344 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:24.152670 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:24.154084 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:24.155170 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:24.155822 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:24.155947 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:24.156667 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:24.156672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:24.156672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:24.156888 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:24.156925 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:24.156940 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:24.157011 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:24.157082 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 08:18:24.157154 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":58,"threads_started":1}
I20260619 08:18:24.158252 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:24.163035 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:24.159658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55527 (local address 127.0.0.1:40233)
0619 08:18:24.159734 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:24.159736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:24.159782 (+ 46us) server_negotiation.cc:407] Connection header received
0619 08:18:24.159844 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:24.159844 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:24.159854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:24.159919 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:24.160037 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:24.160756 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:24.161751 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:24.162048 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:24.162122 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:24.162776 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:24.162778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:24.162778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:24.162782 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:24.162796 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:24.162800 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:24.162856 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:24.162933 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:18:24.162958 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 08:18:24.163367 23240 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:24.163945 23240 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:24.167191 23240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c278d802344691b5f2641133e50b93: Generated new TSK 0
I20260619 08:18:24.167363 23240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:24.169800 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23165
2026-06-19T08:18:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:24 dist-test-slave-d0b5 krb5kdc[23264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:24 dist-test-slave-d0b5 krb5kdc[23264](info): set up 2 sockets
Jun 19 08:18:24 dist-test-slave-d0b5 krb5kdc[23264](info): commencing operation
krb5kdc: starting...
W20260619 08:18:25.210534 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:25 dist-test-slave-d0b5 krb5kdc[23264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:25.328142 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:25.407469 23277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:25.407694 23277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:25.407732 23277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:25.408865 23277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:25.408931 23277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:25.408954 23277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:25.408984 23277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:25.409008 23277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:25.410483 23277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:25.410748 23277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:25.410974 23277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:25.416193 23283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:25.416188 23282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:25.416428 23277 server_base.cc:1034] running on GCE node
W20260619 08:18:25.416446 23285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:25.416715 23277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:25.416970 23277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:25.418144 23277 hybrid_clock.cc:648] HybridClock initialized: now 1781857105418132 us; error 46 us; skew 500 ppm
Jun 19 08:18:25 dist-test-slave-d0b5 krb5kdc[23264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857105, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:25.420739 23277 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:25.421224 23277 webserver.cc:460] Webserver started at http://127.0.0.1:37859/ using document root <none> and password file <none>
I20260619 08:18:25.421463 23277 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:25.421507 23277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:25.421595 23277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:25.422566 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "af75a04a4b7249d6acb0268d962e3401"
format_stamp: "Formatted at 2026-06-19 08:18:25 on dist-test-slave-d0b5"
server_key: "e515cdbb7b47501d041e5b13fc5c483e"
server_key_iv: "26d35c527002dc1e2ed5f93c5f565a96"
server_key_version: "encryptionkey@0"
I20260619 08:18:25.422854 23277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "af75a04a4b7249d6acb0268d962e3401"
format_stamp: "Formatted at 2026-06-19 08:18:25 on dist-test-slave-d0b5"
server_key: "e515cdbb7b47501d041e5b13fc5c483e"
server_key_iv: "26d35c527002dc1e2ed5f93c5f565a96"
server_key_version: "encryptionkey@0"
I20260619 08:18:25.424037 23277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:25.424669 23292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:25.424839 23277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:25.424923 23277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "af75a04a4b7249d6acb0268d962e3401"
format_stamp: "Formatted at 2026-06-19 08:18:25 on dist-test-slave-d0b5"
server_key: "e515cdbb7b47501d041e5b13fc5c483e"
server_key_iv: "26d35c527002dc1e2ed5f93c5f565a96"
server_key_version: "encryptionkey@0"
I20260619 08:18:25.424995 23277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:25.428144 23277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:25.430145 23277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:25.430287 23277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:25.434118 23277 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33635
I20260619 08:18:25.434182 23344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33635 every 8 connection(s)
I20260619 08:18:25.434489 23277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:25.435433 23345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:25.437574 23345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: Bootstrap starting.
I20260619 08:18:25.438073 23345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:25.438294 23345 log.cc:824] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:25.438931 23345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: No bootstrap required, opened a new log
I20260619 08:18:25.440410 23345 raft_consensus.cc:348] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260619 08:18:25.440536 23345 raft_consensus.cc:374] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:25.440558 23345 raft_consensus.cc:729] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af75a04a4b7249d6acb0268d962e3401, State: Initialized, Role: FOLLOWER
I20260619 08:18:25.440654 23345 consensus_queue.cc:260] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260619 08:18:25.440745 23345 raft_consensus.cc:388] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:25.440789 23345 raft_consensus.cc:482] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:25.440857 23345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:25.441430 23345 raft_consensus.cc:504] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260619 08:18:25.441540 23345 leader_election.cc:302] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af75a04a4b7249d6acb0268d962e3401; no voters:
I20260619 08:18:25.441718 23345 leader_election.cc:288] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:25.441776 23348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:25.441957 23345 sys_catalog.cc:564] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:25.441992 23348 raft_consensus.cc:686] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [term 1 LEADER]: Becoming Leader. State: Replica: af75a04a4b7249d6acb0268d962e3401, State: Running, Role: LEADER
I20260619 08:18:25.442080 23348 consensus_queue.cc:237] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } }
I20260619 08:18:25.442487 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23277
I20260619 08:18:25.442622 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:25.442816 2527 external_mini_cluster.cc:1428] Setting key cf3fe791516d7a372e347139d6766214
I20260619 08:18:25.442871 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af75a04a4b7249d6acb0268d962e3401. Latest consensus state: current_term: 1 leader_uuid: "af75a04a4b7249d6acb0268d962e3401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } } }
I20260619 08:18:25.442966 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:25.443009 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af75a04a4b7249d6acb0268d962e3401" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af75a04a4b7249d6acb0268d962e3401" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33635 } } }
I20260619 08:18:25.443063 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401 [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:25.443882 23364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:25.443940 23364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:25.444370 23359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:25.444545 23359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:25.445227 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:25.443424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46361 (local address 127.0.0.1:33635)
0619 08:18:25.443794 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:25.443801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:25.443823 (+ 22us) server_negotiation.cc:407] Connection header received
0619 08:18:25.443913 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:25.443918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:25.443958 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:25.444089 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:25.444801 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:25.444805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:25.444805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:25.444809 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:25.444852 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:25.444869 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:25.444944 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:25.445018 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 08:18:25.445058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":54,"threads_started":1}
I20260619 08:18:25.446187 23359 catalog_manager.cc:1347] Generated new cluster ID: 8d9257b573a24a5cb58a2ab2c96547b8
I20260619 08:18:25.446223 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:25.446239 23359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:25.448709 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:25.447567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36713 (local address 127.0.0.1:33635)
0619 08:18:25.447595 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:25.447597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:25.447723 (+ 126us) server_negotiation.cc:407] Connection header received
0619 08:18:25.447779 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:25.447780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:25.447784 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:25.447858 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:25.448466 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:25.448467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:25.448468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:25.448472 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:25.448487 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:25.448491 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:25.448551 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:25.448627 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 08:18:25.448638 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 08:18:25.461342 23359 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:25.461522 23359 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:25.468084 23359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af75a04a4b7249d6acb0268d962e3401: Generated new TSK 0
I20260619 08:18:25.468256 23359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:25.474303 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23277
2026-06-19T08:18:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:25 dist-test-slave-d0b5 krb5kdc[23376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:25 dist-test-slave-d0b5 krb5kdc[23376](info): set up 2 sockets
Jun 19 08:18:25 dist-test-slave-d0b5 krb5kdc[23376](info): commencing operation
krb5kdc: starting...
W20260619 08:18:26.506326 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:26.629406 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:26.710381 23389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:26.710628 23389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:26.710665 23389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:26.711861 23389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:26.711928 23389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:26.711954 23389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:26.711987 23389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:26.712010 23389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:26.713574 23389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:26.713851 23389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:26.714080 23389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:26.719343 23394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:26.719343 23397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:26.719475 23389 server_base.cc:1034] running on GCE node
W20260619 08:18:26.719403 23395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:26.719868 23389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:26.720176 23389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:26.721366 23389 hybrid_clock.cc:648] HybridClock initialized: now 1781857106721407 us; error 101 us; skew 500 ppm
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:26.723938 23389 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:26.724390 23389 webserver.cc:460] Webserver started at http://127.0.0.1:39873/ using document root <none> and password file <none>
I20260619 08:18:26.724584 23389 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:26.724624 23389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:26.724709 23389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:26.725759 23389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "c3669432d673406f9d7762a725de0d83"
format_stamp: "Formatted at 2026-06-19 08:18:26 on dist-test-slave-d0b5"
server_key: "6d65cbe4f8a565bb9bdbf6faf4c7b4d8"
server_key_iv: "4af77ef83625a8e1978c3238c13535a7"
server_key_version: "encryptionkey@0"
I20260619 08:18:26.726059 23389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "c3669432d673406f9d7762a725de0d83"
format_stamp: "Formatted at 2026-06-19 08:18:26 on dist-test-slave-d0b5"
server_key: "6d65cbe4f8a565bb9bdbf6faf4c7b4d8"
server_key_iv: "4af77ef83625a8e1978c3238c13535a7"
server_key_version: "encryptionkey@0"
I20260619 08:18:26.727303 23389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:26.728053 23404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:26.728282 23389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:26.728365 23389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "c3669432d673406f9d7762a725de0d83"
format_stamp: "Formatted at 2026-06-19 08:18:26 on dist-test-slave-d0b5"
server_key: "6d65cbe4f8a565bb9bdbf6faf4c7b4d8"
server_key_iv: "4af77ef83625a8e1978c3238c13535a7"
server_key_version: "encryptionkey@0"
I20260619 08:18:26.728420 23389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:26.740362 23389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:26.742514 23389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:26.742630 23389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:26.746444 23389 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43595
I20260619 08:18:26.746496 23456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43595 every 8 connection(s)
I20260619 08:18:26.746840 23389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:26.747869 23457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:26.750161 23457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: Bootstrap starting.
I20260619 08:18:26.750761 23457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:26.751056 23457 log.cc:824] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:26.751746 23457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: No bootstrap required, opened a new log
I20260619 08:18:26.753356 23457 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } }
I20260619 08:18:26.753537 23457 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:26.753582 23457 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3669432d673406f9d7762a725de0d83, State: Initialized, Role: FOLLOWER
I20260619 08:18:26.753752 23457 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } }
I20260619 08:18:26.753836 23457 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:26.753876 23457 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:26.753947 23457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:26.754135 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23389
I20260619 08:18:26.754251 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:26.754428 2527 external_mini_cluster.cc:1428] Setting key 474fe1ced28f4f91b1f1dcd0deed9ef2
I20260619 08:18:26.754642 23457 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } }
I20260619 08:18:26.754756 23457 leader_election.cc:302] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3669432d673406f9d7762a725de0d83; no voters:
I20260619 08:18:26.754954 23457 leader_election.cc:288] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:26.755175 23463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:26.755285 23457 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:26.755300 23463 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [term 1 LEADER]: Becoming Leader. State: Replica: c3669432d673406f9d7762a725de0d83, State: Running, Role: LEADER
I20260619 08:18:26.756016 23463 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } }
I20260619 08:18:26.756757 23461 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3669432d673406f9d7762a725de0d83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } } }
I20260619 08:18:26.756757 23464 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3669432d673406f9d7762a725de0d83. Latest consensus state: current_term: 1 leader_uuid: "c3669432d673406f9d7762a725de0d83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3669432d673406f9d7762a725de0d83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43595 } } }
I20260619 08:18:26.756886 23461 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:26.756901 23464 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83 [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:26.757395 23476 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:26.757439 23476 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:26.757494 23477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:26.757684 23477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:26.759292 23477 catalog_manager.cc:1347] Generated new cluster ID: 7de37fa3b1af4f908c00f1c4601831c2
I20260619 08:18:26.759343 23477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:18:26.766175 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:26.754875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:33421 (local address 127.0.0.1:43595)
0619 08:18:26.755048 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:26.755058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:26.755094 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:18:26.755202 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:26.755207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:26.755288 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:26.755420 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:26.756358 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:26.757855 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:26.758837 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:26.759435 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:26.759543 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:26.762407 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:26.762411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:26.762411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:26.762423 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:26.764407 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:26.764892 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:26.764895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:26.764895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:26.765013 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:26.765260 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:26.765261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:26.765261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:26.765572 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:26.765721 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:26.765875 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 08:18:26.765971 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260619 08:18:26.767187 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:26.776067 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:26.768621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:40291 (local address 127.0.0.1:43595)
0619 08:18:26.768661 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:26.768663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:26.768812 (+ 149us) server_negotiation.cc:407] Connection header received
0619 08:18:26.768880 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:26.768881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:26.768891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:26.768943 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:26.769609 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:26.770313 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:26.771362 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:26.771683 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:26.771740 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:26.773154 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:26.773155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:26.773155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:26.773162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:26.774805 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:26.775227 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:26.775228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:26.775228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:26.775283 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:26.775513 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:26.775513 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:26.775514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:26.775636 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:26.775758 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:26.775914 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 08:18:26.775975 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:26.777076 23477 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:26.777599 23477 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:26.785246 23477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3669432d673406f9d7762a725de0d83: Generated new TSK 0
I20260619 08:18:26.785465 23477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:26.790006 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23389
W20260619 08:18:26.795329 23374 connection.cc:535] client connection to 127.0.0.1:43595 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43595): Connection reset by peer (error 104)
2026-06-19T08:18:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23488](info): set up 2 sockets
Jun 19 08:18:26 dist-test-slave-d0b5 krb5kdc[23488](info): commencing operation
krb5kdc: starting...
W20260619 08:18:27.858347 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:27 dist-test-slave-d0b5 krb5kdc[23488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:27.978080 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:28.058305 23501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:28.058543 23501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:28.058579 23501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:28.059732 23501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:28.059804 23501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:28.059829 23501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:28.059870 23501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:28.059895 23501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:28.061398 23501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:28.061666 23501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:28.061893 23501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:28.066920 23506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:28.066936 23507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:28.066985 23509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:28.066972 23501 server_base.cc:1034] running on GCE node
I20260619 08:18:28.067341 23501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:28.067626 23501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:28.068782 23501 hybrid_clock.cc:648] HybridClock initialized: now 1781857108068762 us; error 43 us; skew 500 ppm
Jun 19 08:18:28 dist-test-slave-d0b5 krb5kdc[23488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:28.071352 23501 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:28.071765 23501 webserver.cc:460] Webserver started at http://127.0.0.1:33165/ using document root <none> and password file <none>
I20260619 08:18:28.072028 23501 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:28.072075 23501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:28.072253 23501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:28.073447 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "84b08fabecbd4b63a578388e02972cad"
format_stamp: "Formatted at 2026-06-19 08:18:28 on dist-test-slave-d0b5"
server_key: "0cfeff3a4d166f22c63396a8ce4a4eef"
server_key_iv: "ddacad11520d1ced138965ebc3696970"
server_key_version: "encryptionkey@0"
I20260619 08:18:28.073814 23501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "84b08fabecbd4b63a578388e02972cad"
format_stamp: "Formatted at 2026-06-19 08:18:28 on dist-test-slave-d0b5"
server_key: "0cfeff3a4d166f22c63396a8ce4a4eef"
server_key_iv: "ddacad11520d1ced138965ebc3696970"
server_key_version: "encryptionkey@0"
I20260619 08:18:28.075104 23501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:28.075896 23516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:28.076136 23501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:28.076246 23501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "84b08fabecbd4b63a578388e02972cad"
format_stamp: "Formatted at 2026-06-19 08:18:28 on dist-test-slave-d0b5"
server_key: "0cfeff3a4d166f22c63396a8ce4a4eef"
server_key_iv: "ddacad11520d1ced138965ebc3696970"
server_key_version: "encryptionkey@0"
I20260619 08:18:28.076331 23501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:28.094072 23501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:28.096356 23501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:28.096553 23501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:28.100699 23501 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45951
I20260619 08:18:28.100742 23568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45951 every 8 connection(s)
I20260619 08:18:28.101140 23501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:28.102207 23569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:28.102895 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23501
I20260619 08:18:28.103077 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:28.103308 2527 external_mini_cluster.cc:1428] Setting key 26d4d510673c4508ec19bc82e46064c5
I20260619 08:18:28.105613 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad: Bootstrap starting.
I20260619 08:18:28.106210 23569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:28.106536 23569 log.cc:824] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:28.107316 23569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad: No bootstrap required, opened a new log
I20260619 08:18:28.109346 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:28.103931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:54997 (local address 127.0.0.1:45951)
0619 08:18:28.104119 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:28.104128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:28.104165 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:18:28.104627 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:28.104634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:28.104687 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:28.104828 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:28.105152 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:28.106236 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:28.107340 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:28.107802 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:28.107886 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:28.108615 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:28.108619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:28.108619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:28.108777 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:28.108807 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:28.108817 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:28.108889 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:28.109084 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 08:18:28.109117 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260619 08:18:28.109685 23569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } }
I20260619 08:18:28.109855 23569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:28.109891 23569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b08fabecbd4b63a578388e02972cad, State: Initialized, Role: FOLLOWER
I20260619 08:18:28.110020 23569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } }
I20260619 08:18:28.110090 23569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:28.110129 23569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:28.110178 23569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:28.110491 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:28.110869 23569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } }
I20260619 08:18:28.110985 23569 leader_election.cc:302] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84b08fabecbd4b63a578388e02972cad; no voters:
I20260619 08:18:28.111167 23569 leader_election.cc:288] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:28.111198 23574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:28.111306 23574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [term 1 LEADER]: Becoming Leader. State: Replica: 84b08fabecbd4b63a578388e02972cad, State: Running, Role: LEADER
I20260619 08:18:28.111387 23574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } }
I20260619 08:18:28.111423 23569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:28.111703 23574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84b08fabecbd4b63a578388e02972cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } } }
I20260619 08:18:28.111822 23574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:28.112452 23575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84b08fabecbd4b63a578388e02972cad. Latest consensus state: current_term: 1 leader_uuid: "84b08fabecbd4b63a578388e02972cad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b08fabecbd4b63a578388e02972cad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45951 } } }
I20260619 08:18:28.112592 23575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:28.113112 23585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:28.113663 23585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:28.115352 23585 catalog_manager.cc:1347] Generated new cluster ID: 02a6ebc2d0cc434e92f6a83f277058b1
I20260619 08:18:28.115408 23585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:28.115921 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:28.112132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:41303 (local address 127.0.0.1:45951)
0619 08:18:28.112169 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:28.112171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:28.112319 (+ 148us) server_negotiation.cc:407] Connection header received
0619 08:18:28.112473 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:28.112474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:28.112484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:28.112540 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:28.112731 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:28.113515 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:28.114571 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:28.114842 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:28.114916 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:28.115645 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:28.115646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:28.115647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:28.115651 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:28.115664 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:28.115668 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:28.115741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:28.115843 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 08:18:28.115853 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:28.133960 23585 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:28.134756 23585 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:28.141747 23585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84b08fabecbd4b63a578388e02972cad: Generated new TSK 0
I20260619 08:18:28.141957 23585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:28.143999 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23501
W20260619 08:18:28.147799 23486 connection.cc:535] client connection to 127.0.0.1:45951 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45951): Connection reset by peer (error 104)
2026-06-19T08:18:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:28 dist-test-slave-d0b5 krb5kdc[23600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:28 dist-test-slave-d0b5 krb5kdc[23600](info): set up 2 sockets
Jun 19 08:18:28 dist-test-slave-d0b5 krb5kdc[23600](info): commencing operation
krb5kdc: starting...
W20260619 08:18:29.206499 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:29 dist-test-slave-d0b5 krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:29.324083 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:29.403982 23613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:29.404176 23613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:29.404199 23613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:29.405364 23613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:29.405412 23613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:29.405428 23613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:29.405441 23613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:29.405453 23613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:29.407037 23613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:29.407284 23613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:29.407521 23613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:29.412545 23618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:29.412678 23619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:29.412693 23621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:29.412951 23613 server_base.cc:1034] running on GCE node
I20260619 08:18:29.413223 23613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:29.413583 23613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:29.414724 23613 hybrid_clock.cc:648] HybridClock initialized: now 1781857109414709 us; error 38 us; skew 500 ppm
Jun 19 08:18:29 dist-test-slave-d0b5 krb5kdc[23600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857109, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:29.420971 23613 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:29.421530 23613 webserver.cc:460] Webserver started at http://127.0.0.1:44021/ using document root <none> and password file <none>
I20260619 08:18:29.421766 23613 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:29.421833 23613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:29.422008 23613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:29.423065 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "08b9715579704641a0b80f2e96fec5df"
format_stamp: "Formatted at 2026-06-19 08:18:29 on dist-test-slave-d0b5"
server_key: "802646d7b07d0c6a8b3b596d4f4fbedd"
server_key_iv: "a3caca046d191cd8afc3bdad8955937d"
server_key_version: "encryptionkey@0"
I20260619 08:18:29.423432 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "08b9715579704641a0b80f2e96fec5df"
format_stamp: "Formatted at 2026-06-19 08:18:29 on dist-test-slave-d0b5"
server_key: "802646d7b07d0c6a8b3b596d4f4fbedd"
server_key_iv: "a3caca046d191cd8afc3bdad8955937d"
server_key_version: "encryptionkey@0"
I20260619 08:18:29.424727 23613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:29.425491 23628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:29.425664 23613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:29.425761 23613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "08b9715579704641a0b80f2e96fec5df"
format_stamp: "Formatted at 2026-06-19 08:18:29 on dist-test-slave-d0b5"
server_key: "802646d7b07d0c6a8b3b596d4f4fbedd"
server_key_iv: "a3caca046d191cd8afc3bdad8955937d"
server_key_version: "encryptionkey@0"
I20260619 08:18:29.425843 23613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:29.428989 23613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:29.431227 23613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:29.431382 23613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:29.435520 23613 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43521
I20260619 08:18:29.435566 23680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43521 every 8 connection(s)
I20260619 08:18:29.435921 23613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:29.436865 23681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:29.438333 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23613
I20260619 08:18:29.438476 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:29.438659 2527 external_mini_cluster.cc:1428] Setting key aa0c6cfd9a572640a1117347656594f7
I20260619 08:18:29.439437 23681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df: Bootstrap starting.
I20260619 08:18:29.440047 23681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:29.440423 23681 log.cc:824] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:29.441167 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:29.439187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:49937 (local address 127.0.0.1:43521)
0619 08:18:29.439349 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:29.439355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:29.439379 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:18:29.439822 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:29.439828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:29.439853 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:29.439981 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:29.440676 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:29.440680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:29.440680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:29.440686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:29.440724 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:29.440739 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:29.440819 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:29.440939 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 08:18:29.440983 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260619 08:18:29.441244 23681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df: No bootstrap required, opened a new log
I20260619 08:18:29.442154 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:29.443651 23681 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260619 08:18:29.443864 23681 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:29.443912 23681 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b9715579704641a0b80f2e96fec5df, State: Initialized, Role: FOLLOWER
I20260619 08:18:29.444046 23681 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260619 08:18:29.444146 23681 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:29.444182 23681 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:29.444239 23681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:29.444796 23684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:29.443772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:49239 (local address 127.0.0.1:43521)
0619 08:18:29.443814 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:29.443816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:29.443945 (+ 129us) server_negotiation.cc:407] Connection header received
0619 08:18:29.444058 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:29.444058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:29.444064 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:29.444126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:29.444632 (+ 506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:29.444633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:29.444633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:29.444636 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:29.444648 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:29.444652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:29.444695 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:29.444746 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 08:18:29.444757 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:29.444999 23681 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260619 08:18:29.445107 23681 leader_election.cc:302] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b9715579704641a0b80f2e96fec5df; no voters:
I20260619 08:18:29.445254 23681 leader_election.cc:288] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:29.445379 23691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:29.445533 23691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [term 1 LEADER]: Becoming Leader. State: Replica: 08b9715579704641a0b80f2e96fec5df, State: Running, Role: LEADER
I20260619 08:18:29.445530 23681 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:29.445645 23691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260619 08:18:29.446113 23693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b9715579704641a0b80f2e96fec5df. Latest consensus state: current_term: 1 leader_uuid: "08b9715579704641a0b80f2e96fec5df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260619 08:18:29.446209 23693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:29.446630 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08b9715579704641a0b80f2e96fec5df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9715579704641a0b80f2e96fec5df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260619 08:18:29.446743 23700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:29.446858 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:29.447366 23700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:29.448889 23700 catalog_manager.cc:1347] Generated new cluster ID: 3af7041ac8b54a5bbb1d9ed5a7d0ecd7
I20260619 08:18:29.448940 23700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:29.455008 23700 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:29.455171 23700 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:29.459740 23700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08b9715579704641a0b80f2e96fec5df: Generated new TSK 0
I20260619 08:18:29.459910 23700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:29.462090 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23613
2026-06-19T08:18:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:29 dist-test-slave-d0b5 krb5kdc[23712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:29 dist-test-slave-d0b5 krb5kdc[23712](info): set up 2 sockets
Jun 19 08:18:29 dist-test-slave-d0b5 krb5kdc[23712](info): commencing operation
krb5kdc: starting...
W20260619 08:18:30.502404 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:30.624580 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:30.704440 23725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:30.704690 23725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:30.704741 23725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:30.705914 23725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:30.705991 23725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:30.706019 23725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:30.706051 23725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:30.706075 23725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:30.707551 23725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43255
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:30.707837 23725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:30.708071 23725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:18:30.713195 23725 server_base.cc:1034] running on GCE node
W20260619 08:18:30.713160 23733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:30.713140 23730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:30.713161 23731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:30.713668 23725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:30.713953 23725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:30.715161 23725 hybrid_clock.cc:648] HybridClock initialized: now 1781857110715143 us; error 28 us; skew 500 ppm
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:30.717579 23725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:30.718011 23725 webserver.cc:460] Webserver started at http://127.0.0.1:35655/ using document root <none> and password file <none>
I20260619 08:18:30.718205 23725 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:30.718245 23725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:30.718340 23725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:30.719347 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "b121248c2bc74761a6793258b5bdecae"
format_stamp: "Formatted at 2026-06-19 08:18:30 on dist-test-slave-d0b5"
server_key: "150d20f6e8801516458e84f6abc5b3f5"
server_key_iv: "e46c3459f0ecc70fcb04a893c8ed26a0"
server_key_version: "encryptionkey@0"
I20260619 08:18:30.719636 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "b121248c2bc74761a6793258b5bdecae"
format_stamp: "Formatted at 2026-06-19 08:18:30 on dist-test-slave-d0b5"
server_key: "150d20f6e8801516458e84f6abc5b3f5"
server_key_iv: "e46c3459f0ecc70fcb04a893c8ed26a0"
server_key_version: "encryptionkey@0"
I20260619 08:18:30.720883 23725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:30.721591 23740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:30.721889 23725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:30.721983 23725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "b121248c2bc74761a6793258b5bdecae"
format_stamp: "Formatted at 2026-06-19 08:18:30 on dist-test-slave-d0b5"
server_key: "150d20f6e8801516458e84f6abc5b3f5"
server_key_iv: "e46c3459f0ecc70fcb04a893c8ed26a0"
server_key_version: "encryptionkey@0"
I20260619 08:18:30.722031 23725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:30.732362 23725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:30.734752 23725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:30.734865 23725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:30.738582 23725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43255
I20260619 08:18:30.738639 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43255 every 8 connection(s)
I20260619 08:18:30.738951 23725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:30.739192 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23725
I20260619 08:18:30.739318 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:30.739574 2527 external_mini_cluster.cc:1428] Setting key 3f270adcc2aa3f3c6fa4aedc81ef99df
I20260619 08:18:30.740193 23793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:30.743394 23793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: Bootstrap starting.
I20260619 08:18:30.744014 23793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:30.744318 23793 log.cc:824] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:30.745047 23793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: No bootstrap required, opened a new log
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 08:18:30.746923 23793 raft_consensus.cc:348] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } }
I20260619 08:18:30.747073 23793 raft_consensus.cc:374] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:30.747123 23793 raft_consensus.cc:729] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b121248c2bc74761a6793258b5bdecae, State: Initialized, Role: FOLLOWER
I20260619 08:18:30.747274 23793 consensus_queue.cc:260] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } }
I20260619 08:18:30.747391 23793 raft_consensus.cc:388] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:30.747426 23793 raft_consensus.cc:482] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:30.747469 23793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:30.748086 23793 raft_consensus.cc:504] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } }
I20260619 08:18:30.748185 23793 leader_election.cc:302] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b121248c2bc74761a6793258b5bdecae; no voters:
I20260619 08:18:30.748384 23793 leader_election.cc:288] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:30.748438 23798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:30.748597 23798 raft_consensus.cc:686] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [term 1 LEADER]: Becoming Leader. State: Replica: b121248c2bc74761a6793258b5bdecae, State: Running, Role: LEADER
I20260619 08:18:30.748668 23793 sys_catalog.cc:564] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:30.748680 23798 consensus_queue.cc:237] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } }
I20260619 08:18:30.750931 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:30.740106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:39981 (local address 127.0.0.1:43255)
0619 08:18:30.740329 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:30.740339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:30.740378 (+ 39us) server_negotiation.cc:407] Connection header received
0619 08:18:30.740928 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:30.740938 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:30.741003 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:30.741164 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:30.741864 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:30.743220 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:30.744022 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:30.744631 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:30.744766 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:30.747070 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:30.747074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:30.747074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:30.747086 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:30.749086 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:30.749540 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:30.749543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:30.749543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:30.749656 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:30.749908 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:30.749909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:30.749909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:30.750235 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:30.750395 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:30.750610 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 08:18:30.750717 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":63,"threads_started":1}
I20260619 08:18:30.750087 23799 sys_catalog.cc:455] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b121248c2bc74761a6793258b5bdecae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } } }
I20260619 08:18:30.751147 23799 sys_catalog.cc:458] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:30.750227 23800 sys_catalog.cc:455] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b121248c2bc74761a6793258b5bdecae. Latest consensus state: current_term: 1 leader_uuid: "b121248c2bc74761a6793258b5bdecae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b121248c2bc74761a6793258b5bdecae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43255 } } }
I20260619 08:18:30.751302 23800 sys_catalog.cc:458] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:30.751421 23812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:30.751464 23812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:30.751609 23814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:30.751792 23814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:30.751850 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:30.756847 23814 catalog_manager.cc:1347] Generated new cluster ID: ec8f929dbe0140ab8abdf6c5c7768e64
I20260619 08:18:30.756924 23814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:30.760154 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:30.753331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:60297 (local address 127.0.0.1:43255)
0619 08:18:30.753378 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:30.753380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:30.753473 (+ 93us) server_negotiation.cc:407] Connection header received
0619 08:18:30.753542 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:30.753542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:30.753550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:30.753591 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:30.754221 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:30.754679 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:30.755705 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:30.755931 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:30.755981 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:30.757441 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:30.757443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:30.757443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:30.757449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:30.759002 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:30.759380 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:30.759381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:30.759381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:30.759427 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:30.759625 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:30.759625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:30.759626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:30.759739 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:30.759804 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:30.760010 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 08:18:30.760070 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:30.771363 23814 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:30.771938 23814 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:30.783634 23814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b121248c2bc74761a6793258b5bdecae: Generated new TSK 0
I20260619 08:18:30.783855 23814 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:30.790529 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23725
W20260619 08:18:30.795084 23710 connection.cc:535] client connection to 127.0.0.1:43255 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43255): Connection reset by peer (error 104)
2026-06-19T08:18:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23824](info): set up 2 sockets
Jun 19 08:18:30 dist-test-slave-d0b5 krb5kdc[23824](info): commencing operation
krb5kdc: starting...
W20260619 08:18:31.834465 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:31 dist-test-slave-d0b5 krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:31.952729 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:32.033591 23837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:32.033815 23837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:32.033854 23837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:32.035019 23837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:32.035089 23837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:32.035115 23837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:32.035143 23837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:32.035167 23837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:32.036671 23837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:32.036954 23837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:32.037177 23837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:32.042069 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:32.042083 23842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:32.042086 23843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:32.042387 23837 server_base.cc:1034] running on GCE node
I20260619 08:18:32.042626 23837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:32.042922 23837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:32.044094 23837 hybrid_clock.cc:648] HybridClock initialized: now 1781857112044069 us; error 43 us; skew 500 ppm
Jun 19 08:18:32 dist-test-slave-d0b5 krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:32.046761 23837 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:32.047279 23837 webserver.cc:460] Webserver started at http://127.0.0.1:40231/ using document root <none> and password file <none>
I20260619 08:18:32.047501 23837 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:32.047564 23837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:32.047711 23837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:32.048840 23837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "8804dc77ff4c421c9dc4fca71a8aa54a"
format_stamp: "Formatted at 2026-06-19 08:18:32 on dist-test-slave-d0b5"
server_key: "7e617f3bdd26df43de2e76033edb00a7"
server_key_iv: "ab6c89f12819ba7286f513e00c0cac82"
server_key_version: "encryptionkey@0"
I20260619 08:18:32.049209 23837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "8804dc77ff4c421c9dc4fca71a8aa54a"
format_stamp: "Formatted at 2026-06-19 08:18:32 on dist-test-slave-d0b5"
server_key: "7e617f3bdd26df43de2e76033edb00a7"
server_key_iv: "ab6c89f12819ba7286f513e00c0cac82"
server_key_version: "encryptionkey@0"
I20260619 08:18:32.050570 23837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:32.051374 23852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:32.051569 23837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:32.051664 23837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "8804dc77ff4c421c9dc4fca71a8aa54a"
format_stamp: "Formatted at 2026-06-19 08:18:32 on dist-test-slave-d0b5"
server_key: "7e617f3bdd26df43de2e76033edb00a7"
server_key_iv: "ab6c89f12819ba7286f513e00c0cac82"
server_key_version: "encryptionkey@0"
I20260619 08:18:32.051740 23837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:32.059283 23837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:32.061606 23837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:32.061761 23837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:32.065629 23837 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38101
I20260619 08:18:32.065757 23904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38101 every 8 connection(s)
I20260619 08:18:32.066056 23837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:32.066980 23905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:32.067394 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23837
I20260619 08:18:32.067536 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:32.067768 2527 external_mini_cluster.cc:1428] Setting key 544b5511f70cf569f4045c2914f12a8d
I20260619 08:18:32.070142 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: Bootstrap starting.
I20260619 08:18:32.070834 23905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:32.071170 23905 log.cc:824] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:32.071894 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: No bootstrap required, opened a new log
I20260619 08:18:32.073969 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:32.068342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:40043 (local address 127.0.0.1:38101)
0619 08:18:32.068535 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:32.068545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:32.068587 (+ 42us) server_negotiation.cc:407] Connection header received
0619 08:18:32.069115 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:32.069123 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:32.069184 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:32.069363 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:32.069610 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:32.071025 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:32.071848 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:32.072426 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:32.072538 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:32.073200 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:32.073204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:32.073205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:32.073431 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:32.073469 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:32.073484 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:32.073581 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:32.073668 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:18:32.073699 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260619 08:18:32.074254 23905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } }
I20260619 08:18:32.074402 23905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:32.074447 23905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8804dc77ff4c421c9dc4fca71a8aa54a, State: Initialized, Role: FOLLOWER
I20260619 08:18:32.074574 23905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } }
I20260619 08:18:32.074683 23905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:32.074731 23905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:32.074775 23905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:32.075039 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:32.075431 23905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } }
I20260619 08:18:32.075549 23905 leader_election.cc:302] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8804dc77ff4c421c9dc4fca71a8aa54a; no voters:
I20260619 08:18:32.075733 23905 leader_election.cc:288] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:32.075804 23910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:32.075971 23910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [term 1 LEADER]: Becoming Leader. State: Replica: 8804dc77ff4c421c9dc4fca71a8aa54a, State: Running, Role: LEADER
I20260619 08:18:32.075989 23905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:32.076738 23910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } }
I20260619 08:18:32.077349 23912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8804dc77ff4c421c9dc4fca71a8aa54a. Latest consensus state: current_term: 1 leader_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } } }
I20260619 08:18:32.077433 23912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:32.077661 23911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8804dc77ff4c421c9dc4fca71a8aa54a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38101 } } }
I20260619 08:18:32.077723 23911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:32.078087 23929 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:32.078184 23929 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:32.078269 23930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:32.078421 23930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:32.080008 23930 catalog_manager.cc:1347] Generated new cluster ID: 132b499c30e64eabb43f3f4b471d1a63
I20260619 08:18:32.080101 23930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:32.080247 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:32.076546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:57245 (local address 127.0.0.1:38101)
0619 08:18:32.076582 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:32.076584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:32.076661 (+ 77us) server_negotiation.cc:407] Connection header received
0619 08:18:32.076747 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:32.076748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:32.076759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:32.076823 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:32.077106 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:32.077866 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:32.078860 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:32.079162 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:32.079224 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:32.079995 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:32.079996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:32.079996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:32.080000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:32.080013 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:32.080017 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:32.080068 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:32.080137 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 08:18:32.080146 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:32.109920 23930 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:32.110545 23930 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:32.123474 23930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8804dc77ff4c421c9dc4fca71a8aa54a: Generated new TSK 0
I20260619 08:18:32.123700 23930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:32.128553 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23837
W20260619 08:18:32.132953 23822 connection.cc:535] client connection to 127.0.0.1:38101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38101): Connection reset by peer (error 104)
2026-06-19T08:18:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:32 dist-test-slave-d0b5 krb5kdc[23936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:32 dist-test-slave-d0b5 krb5kdc[23936](info): set up 2 sockets
Jun 19 08:18:32 dist-test-slave-d0b5 krb5kdc[23936](info): commencing operation
krb5kdc: starting...
W20260619 08:18:33.166347 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:33 dist-test-slave-d0b5 krb5kdc[23936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 08:18:33.292640 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:33.371850 23949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:33.372099 23949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:33.372135 23949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:33.373322 23949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:33.373404 23949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:33.373431 23949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:33.373464 23949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:33.373486 23949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:33.374969 23949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:33.375247 23949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:33.375485 23949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:18:33.380437 23949 server_base.cc:1034] running on GCE node
W20260619 08:18:33.380419 23955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:33.380411 23954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:33.380419 23957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:33.380825 23949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:33.381130 23949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:33.382333 23949 hybrid_clock.cc:648] HybridClock initialized: now 1781857113382296 us; error 48 us; skew 500 ppm
Jun 19 08:18:33 dist-test-slave-d0b5 krb5kdc[23936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857113, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:33.384691 23949 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 08:18:33.385120 23949 webserver.cc:460] Webserver started at http://127.0.0.1:37161/ using document root <none> and password file <none>
I20260619 08:18:33.385589 23949 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:33.385677 23949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:33.385825 23949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:33.386888 23949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "ab5c4e3a00744e1282012ea7acef98db"
format_stamp: "Formatted at 2026-06-19 08:18:33 on dist-test-slave-d0b5"
server_key: "dcaa9644ebf06d8cc87c43fdaefb463f"
server_key_iv: "270feb6ebaa27eab50d065407fdc6dfe"
server_key_version: "encryptionkey@0"
I20260619 08:18:33.387274 23949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "ab5c4e3a00744e1282012ea7acef98db"
format_stamp: "Formatted at 2026-06-19 08:18:33 on dist-test-slave-d0b5"
server_key: "dcaa9644ebf06d8cc87c43fdaefb463f"
server_key_iv: "270feb6ebaa27eab50d065407fdc6dfe"
server_key_version: "encryptionkey@0"
I20260619 08:18:33.388751 23949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:33.389578 23964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:33.389747 23949 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:33.389848 23949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "ab5c4e3a00744e1282012ea7acef98db"
format_stamp: "Formatted at 2026-06-19 08:18:33 on dist-test-slave-d0b5"
server_key: "dcaa9644ebf06d8cc87c43fdaefb463f"
server_key_iv: "270feb6ebaa27eab50d065407fdc6dfe"
server_key_version: "encryptionkey@0"
I20260619 08:18:33.389935 23949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:33.393097 23949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:33.395017 23949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:33.395170 23949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:33.399235 23949 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40781
I20260619 08:18:33.399282 24016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40781 every 8 connection(s)
I20260619 08:18:33.399592 23949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:33.400311 24017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:33.402527 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db: Bootstrap starting.
I20260619 08:18:33.403101 24017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:33.403371 24017 log.cc:824] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:33.404076 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db: No bootstrap required, opened a new log
I20260619 08:18:33.405740 24017 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } }
I20260619 08:18:33.405885 24017 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:33.405910 24017 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5c4e3a00744e1282012ea7acef98db, State: Initialized, Role: FOLLOWER
I20260619 08:18:33.406077 24017 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } }
I20260619 08:18:33.406159 24017 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:33.406201 24017 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:33.406258 24017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:33.406776 24017 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } }
I20260619 08:18:33.406883 24017 leader_election.cc:302] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab5c4e3a00744e1282012ea7acef98db; no voters:
I20260619 08:18:33.407068 24017 leader_election.cc:288] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:33.407163 24020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:33.407321 24017 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:33.407379 24020 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [term 1 LEADER]: Becoming Leader. State: Replica: ab5c4e3a00744e1282012ea7acef98db, State: Running, Role: LEADER
I20260619 08:18:33.407528 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 23949
I20260619 08:18:33.407567 24020 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } }
I20260619 08:18:33.407660 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:33.407869 2527 external_mini_cluster.cc:1428] Setting key f680bc6ec1da47a6e25669d784d16c15
I20260619 08:18:33.408108 24022 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab5c4e3a00744e1282012ea7acef98db. Latest consensus state: current_term: 1 leader_uuid: "ab5c4e3a00744e1282012ea7acef98db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } } }
I20260619 08:18:33.408193 24022 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:33.408564 24021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab5c4e3a00744e1282012ea7acef98db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab5c4e3a00744e1282012ea7acef98db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40781 } } }
I20260619 08:18:33.408689 24021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:33.408713 24024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:33.409363 24024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:33.410632 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:33.408448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:37195 (local address 127.0.0.1:40781)
0619 08:18:33.408723 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:33.408731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:33.408762 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:18:33.408885 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:33.408893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:33.408928 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:33.409400 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:33.409867 (+ 467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:33.409870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:33.409871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:33.409874 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:33.409914 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:33.409935 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:33.410011 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:33.410428 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 08:18:33.410471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":74,"threads_started":1}
I20260619 08:18:33.410970 24024 catalog_manager.cc:1347] Generated new cluster ID: 8fb6db2dfdaf4bdc97e8545ed2cbb7cc
I20260619 08:18:33.411023 24024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:33.411643 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:33.418560 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:33.413262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.233:40595 (local address 127.0.0.1:40781)
0619 08:18:33.413317 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:33.413319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:33.417506 (+ 4187us) server_negotiation.cc:407] Connection header received
0619 08:18:33.417584 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:33.417585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:33.417590 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:33.417642 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:33.418321 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:33.418322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:33.418323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:33.418327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:33.418342 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:33.418347 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:33.418392 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:33.418456 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:18:33.418471 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 08:18:33.431433 24024 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:33.431614 24024 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:33.447002 24024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab5c4e3a00744e1282012ea7acef98db: Generated new TSK 0
I20260619 08:18:33.447247 24024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:33.449570 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 23949
2026-06-19T08:18:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:33 dist-test-slave-d0b5 krb5kdc[24048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:33 dist-test-slave-d0b5 krb5kdc[24048](info): set up 2 sockets
Jun 19 08:18:33 dist-test-slave-d0b5 krb5kdc[24048](info): commencing operation
krb5kdc: starting...
W20260619 08:18:34.490429 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:34.616530 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:35539
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:35539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:34.697121 24061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:34.697338 24061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:34.697361 24061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:34.698523 24061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:34.698565 24061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:34.698580 24061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:34.698592 24061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:34.698604 24061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:34.700068 24061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:35539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:35539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:34.700309 24061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:34.700551 24061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:34.705389 24066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:34.705407 24069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:34.705456 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:34.705557 24061 server_base.cc:1034] running on GCE node
I20260619 08:18:34.705899 24061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:34.706183 24061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:34.707368 24061 hybrid_clock.cc:648] HybridClock initialized: now 1781857114707344 us; error 55 us; skew 500 ppm
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857114, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:34.712914 24061 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:34.713361 24061 webserver.cc:460] Webserver started at http://127.2.119.254:35663/ using document root <none> and password file <none>
I20260619 08:18:34.713581 24061 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:34.713626 24061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:34.713758 24061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:34.714799 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "0c352a73bbd5487587d7fdd282d9f5e9"
format_stamp: "Formatted at 2026-06-19 08:18:34 on dist-test-slave-d0b5"
server_key: "c8838f84a7e911688b0ffdf0ef5f92c3"
server_key_iv: "6dac93444cf9bddbbe101a321e4cf207"
server_key_version: "encryptionkey@0"
I20260619 08:18:34.715153 24061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "0c352a73bbd5487587d7fdd282d9f5e9"
format_stamp: "Formatted at 2026-06-19 08:18:34 on dist-test-slave-d0b5"
server_key: "c8838f84a7e911688b0ffdf0ef5f92c3"
server_key_iv: "6dac93444cf9bddbbe101a321e4cf207"
server_key_version: "encryptionkey@0"
I20260619 08:18:34.716514 24061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:34.717258 24076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:34.717485 24061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:34.717579 24061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "0c352a73bbd5487587d7fdd282d9f5e9"
format_stamp: "Formatted at 2026-06-19 08:18:34 on dist-test-slave-d0b5"
server_key: "c8838f84a7e911688b0ffdf0ef5f92c3"
server_key_iv: "6dac93444cf9bddbbe101a321e4cf207"
server_key_version: "encryptionkey@0"
I20260619 08:18:34.717635 24061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:34.744608 24061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:34.746706 24061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:34.746816 24061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:34.750607 24061 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:35539
I20260619 08:18:34.750684 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:35539 every 8 connection(s)
I20260619 08:18:34.751032 24061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:34.751488 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24061
I20260619 08:18:34.751606 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:34.751797 2527 external_mini_cluster.cc:1428] Setting key e2a9a5ae8dc33b42a125d7dac575b8e9
I20260619 08:18:34.752123 24129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:34.755646 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9: Bootstrap starting.
I20260619 08:18:34.756287 24129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:34.756598 24129 log.cc:824] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:34.757476 24129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9: No bootstrap required, opened a new log
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:34.759522 24129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } }
I20260619 08:18:34.759701 24129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:34.759769 24129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c352a73bbd5487587d7fdd282d9f5e9, State: Initialized, Role: FOLLOWER
I20260619 08:18:34.759948 24129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } }
I20260619 08:18:34.760036 24129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:34.760080 24129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:34.760129 24129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:34.760638 24129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } }
I20260619 08:18:34.760759 24129 leader_election.cc:302] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c352a73bbd5487587d7fdd282d9f5e9; no voters:
I20260619 08:18:34.760960 24129 leader_election.cc:288] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:34.760993 24134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:34.761109 24134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [term 1 LEADER]: Becoming Leader. State: Replica: 0c352a73bbd5487587d7fdd282d9f5e9, State: Running, Role: LEADER
I20260619 08:18:34.761236 24129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:34.761253 24134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } }
I20260619 08:18:34.761667 24134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c352a73bbd5487587d7fdd282d9f5e9. Latest consensus state: current_term: 1 leader_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } } }
I20260619 08:18:34.761924 24134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:34.761843 24135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c352a73bbd5487587d7fdd282d9f5e9" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 35539 } } }
I20260619 08:18:34.762341 24135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:34.762506 24143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:34.763221 24143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:34.763530 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:34.752350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33707 (local address 127.2.119.254:35539)
0619 08:18:34.752610 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:34.752619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:34.752650 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:18:34.753136 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:34.753144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:34.753206 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:34.753373 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:34.753986 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:34.755368 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:34.756318 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:34.756914 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:34.756998 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:34.759418 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:34.759423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:34.759423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:34.759435 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:34.761631 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:34.762141 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:34.762143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:34.762143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:34.762238 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:34.762516 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:34.762517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:34.762517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:34.762857 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:34.763015 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:34.763209 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 08:18:34.763322 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":154,"threads_started":1}
I20260619 08:18:34.764506 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:34.765687 24143 catalog_manager.cc:1347] Generated new cluster ID: d5c31937032046caba94ddc39fbd822b
I20260619 08:18:34.765743 24143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:34.771924 24143 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:34.772574 24143 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:34.773161 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:34.765980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52211 (local address 127.2.119.254:35539)
0619 08:18:34.766016 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:34.766018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:34.766027 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:34.766068 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:34.766069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:34.766077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:34.766143 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:34.766707 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:34.767445 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:34.768459 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:34.768796 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:34.768870 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:34.770232 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:34.770233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:34.770234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:34.770241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:34.771854 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:34.772287 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:34.772288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:34.772289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:34.772348 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:34.772608 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:34.772609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:34.772610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:34.772730 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:34.772831 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:34.773010 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 08:18:34.773072 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:34.780946 24143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c352a73bbd5487587d7fdd282d9f5e9: Generated new TSK 0
I20260619 08:18:34.781153 24143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:34.788062 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24061
W20260619 08:18:34.792057 24046 connection.cc:535] client connection to 127.2.119.254:35539 recv error: Network error: failed to read from TLS socket (remote: 127.2.119.254:35539): Connection reset by peer (error 104)
2026-06-19T08:18:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24160](info): set up 2 sockets
Jun 19 08:18:34 dist-test-slave-d0b5 krb5kdc[24160](info): commencing operation
krb5kdc: starting...
W20260619 08:18:35.830459 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:35 dist-test-slave-d0b5 krb5kdc[24160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:35.957007 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41943
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:36.037587 24173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:36.037873 24173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:36.037921 24173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:36.039086 24173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:36.039160 24173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:36.039183 24173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:36.039212 24173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:36.039242 24173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:36.040701 24173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41943
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:36.040988 24173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:36.041213 24173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:36.046301 24181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:36.046305 24179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:36.046335 24173 server_base.cc:1034] running on GCE node
W20260619 08:18:36.046305 24178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:36.046904 24173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:36.047156 24173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:36.048328 24173 hybrid_clock.cc:648] HybridClock initialized: now 1781857116048314 us; error 29 us; skew 500 ppm
Jun 19 08:18:36 dist-test-slave-d0b5 krb5kdc[24160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857116, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:36.051038 24173 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:36.051517 24173 webserver.cc:460] Webserver started at http://127.2.119.254:43443/ using document root <none> and password file <none>
I20260619 08:18:36.051757 24173 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:36.051805 24173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:36.051898 24173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:36.053114 24173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "9bc271afc6af47f29d84908fd5471dbf"
format_stamp: "Formatted at 2026-06-19 08:18:36 on dist-test-slave-d0b5"
server_key: "27712bdc6a563e046aa6b562d0dbaef6"
server_key_iv: "dd383274b7fc52b6afdcf4ccfc047591"
server_key_version: "encryptionkey@0"
I20260619 08:18:36.053530 24173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "9bc271afc6af47f29d84908fd5471dbf"
format_stamp: "Formatted at 2026-06-19 08:18:36 on dist-test-slave-d0b5"
server_key: "27712bdc6a563e046aa6b562d0dbaef6"
server_key_iv: "dd383274b7fc52b6afdcf4ccfc047591"
server_key_version: "encryptionkey@0"
I20260619 08:18:36.054937 24173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:36.055809 24188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:36.056075 24173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:36.056178 24173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "9bc271afc6af47f29d84908fd5471dbf"
format_stamp: "Formatted at 2026-06-19 08:18:36 on dist-test-slave-d0b5"
server_key: "27712bdc6a563e046aa6b562d0dbaef6"
server_key_iv: "dd383274b7fc52b6afdcf4ccfc047591"
server_key_version: "encryptionkey@0"
I20260619 08:18:36.056253 24173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:36.079391 24173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:36.081786 24173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:36.081954 24173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:36.086035 24173 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41943
I20260619 08:18:36.086087 24240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41943 every 8 connection(s)
I20260619 08:18:36.086403 24173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:36.087250 24241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:36.089541 24241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf: Bootstrap starting.
I20260619 08:18:36.090065 24241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:36.090291 24241 log.cc:824] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:36.090960 24241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf: No bootstrap required, opened a new log
I20260619 08:18:36.092374 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24173
I20260619 08:18:36.092495 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:36.092694 2527 external_mini_cluster.cc:1428] Setting key 0d5b01f6407c142e408c9f48faf184dc
I20260619 08:18:36.092847 24241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } }
I20260619 08:18:36.093027 24241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:36.093070 24241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bc271afc6af47f29d84908fd5471dbf, State: Initialized, Role: FOLLOWER
I20260619 08:18:36.093502 24241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } }
I20260619 08:18:36.093592 24241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:36.093616 24241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:36.093678 24241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:36.094594 24241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } }
I20260619 08:18:36.094682 24241 leader_election.cc:302] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bc271afc6af47f29d84908fd5471dbf; no voters:
I20260619 08:18:36.094882 24241 leader_election.cc:288] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:36.094902 24246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:36.095006 24246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [term 1 LEADER]: Becoming Leader. State: Replica: 9bc271afc6af47f29d84908fd5471dbf, State: Running, Role: LEADER
I20260619 08:18:36.095098 24246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } }
I20260619 08:18:36.095136 24241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:36.095449 24247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bc271afc6af47f29d84908fd5471dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } } }
I20260619 08:18:36.095541 24247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:36.095721 24248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bc271afc6af47f29d84908fd5471dbf. Latest consensus state: current_term: 1 leader_uuid: "9bc271afc6af47f29d84908fd5471dbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bc271afc6af47f29d84908fd5471dbf" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41943 } } }
I20260619 08:18:36.095796 24248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:36.095890 24251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:36.096421 24251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:36.098567 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:36.093221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43865 (local address 127.2.119.254:41943)
0619 08:18:36.093445 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:36.093453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:36.093479 (+ 26us) server_negotiation.cc:407] Connection header received
0619 08:18:36.093571 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:36.093576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:36.093650 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:36.093822 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:36.094103 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:36.095267 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:36.096460 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:36.096990 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:36.097110 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:36.097870 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:36.097874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:36.097874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:36.098093 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:36.098134 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:36.098155 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:36.098253 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:36.098353 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 08:18:36.098386 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":128,"threads_started":1}
I20260619 08:18:36.099756 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:36.104704 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:36.101231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56007 (local address 127.2.119.254:41943)
0619 08:18:36.101316 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:36.101318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:36.101475 (+ 157us) server_negotiation.cc:407] Connection header received
0619 08:18:36.101532 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:36.101533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:36.101542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:36.101588 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:36.101791 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:36.102348 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:36.103439 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:36.103630 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:36.103685 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:36.104381 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:36.104383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:36.104383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:36.104387 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:36.104399 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:36.104403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:36.104452 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:36.104561 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 08:18:36.104603 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 08:18:36.105677 24251 catalog_manager.cc:1347] Generated new cluster ID: 2395e38744c6410690358ca9eed07c3d
I20260619 08:18:36.105731 24251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:36.117353 24251 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:36.118000 24251 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:36.120669 24251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bc271afc6af47f29d84908fd5471dbf: Generated new TSK 0
I20260619 08:18:36.120874 24251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:36.125841 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24173
W20260619 08:18:36.130785 24158 connection.cc:535] client connection to 127.2.119.254:41943 recv error: Network error: failed to read from TLS socket (remote: 127.2.119.254:41943): Connection reset by peer (error 104)
2026-06-19T08:18:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:36 dist-test-slave-d0b5 krb5kdc[24272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:36 dist-test-slave-d0b5 krb5kdc[24272](info): set up 2 sockets
Jun 19 08:18:36 dist-test-slave-d0b5 krb5kdc[24272](info): commencing operation
krb5kdc: starting...
W20260619 08:18:37.171167 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:37 dist-test-slave-d0b5 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:37.292785 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39963
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:37.373637 24285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:37.373886 24285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:37.373922 24285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:37.375108 24285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:37.375180 24285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:37.375213 24285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:37.375244 24285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:37.375268 24285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:37.376780 24285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:39963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:37.377058 24285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:37.377318 24285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:37.382411 24291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:37.382462 24285 server_base.cc:1034] running on GCE node
W20260619 08:18:37.382483 24290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:37.382515 24293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:37.382851 24285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:37.383224 24285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:37.384428 24285 hybrid_clock.cc:648] HybridClock initialized: now 1781857117384416 us; error 31 us; skew 500 ppm
Jun 19 08:18:37 dist-test-slave-d0b5 krb5kdc[24272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857117, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:37.387015 24285 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:37.387526 24285 webserver.cc:460] Webserver started at http://127.2.119.254:37169/ using document root <none> and password file <none>
I20260619 08:18:37.387773 24285 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:37.387842 24285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:37.387992 24285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:37.389048 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "f7937ed4e69f4c0ca16b38d458fdaa33"
format_stamp: "Formatted at 2026-06-19 08:18:37 on dist-test-slave-d0b5"
server_key: "86c68367939849313ce3a3b6df5ef2dd"
server_key_iv: "3335af20df764d833399d8b0d7466803"
server_key_version: "encryptionkey@0"
I20260619 08:18:37.389459 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "f7937ed4e69f4c0ca16b38d458fdaa33"
format_stamp: "Formatted at 2026-06-19 08:18:37 on dist-test-slave-d0b5"
server_key: "86c68367939849313ce3a3b6df5ef2dd"
server_key_iv: "3335af20df764d833399d8b0d7466803"
server_key_version: "encryptionkey@0"
I20260619 08:18:37.390890 24285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:37.391635 24300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:37.391894 24285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:37.392009 24285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "f7937ed4e69f4c0ca16b38d458fdaa33"
format_stamp: "Formatted at 2026-06-19 08:18:37 on dist-test-slave-d0b5"
server_key: "86c68367939849313ce3a3b6df5ef2dd"
server_key_iv: "3335af20df764d833399d8b0d7466803"
server_key_version: "encryptionkey@0"
I20260619 08:18:37.392086 24285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:37.395485 24285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:37.397509 24285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:37.397662 24285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:37.401811 24285 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:39963
I20260619 08:18:37.401882 24352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:39963 every 8 connection(s)
I20260619 08:18:37.402213 24285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:37.403187 24353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:37.405906 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33: Bootstrap starting.
I20260619 08:18:37.406477 24353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:37.406780 24353 log.cc:824] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:37.407516 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33: No bootstrap required, opened a new log
I20260619 08:18:37.407672 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24285
I20260619 08:18:37.407806 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:37.408036 2527 external_mini_cluster.cc:1428] Setting key aceca94db9b2631b16c9899cf574d8f7
I20260619 08:18:37.410346 24353 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } }
I20260619 08:18:37.410521 24353 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:37.410558 24353 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7937ed4e69f4c0ca16b38d458fdaa33, State: Initialized, Role: FOLLOWER
I20260619 08:18:37.410778 24353 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } }
I20260619 08:18:37.410884 24353 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:37.410938 24353 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:37.410985 24353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:37.411718 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:37.408589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47617 (local address 127.2.119.254:39963)
0619 08:18:37.408770 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:37.408779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:37.408816 (+ 37us) server_negotiation.cc:407] Connection header received
0619 08:18:37.410295 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:37.410305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:37.410341 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:37.410490 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:37.411266 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:37.411269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:37.411270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:37.411274 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:37.411316 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:37.411330 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:37.411402 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:37.411457 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 08:18:37.411500 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260619 08:18:37.411716 24353 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } }
I20260619 08:18:37.411860 24353 leader_election.cc:302] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7937ed4e69f4c0ca16b38d458fdaa33; no voters:
I20260619 08:18:37.412067 24353 leader_election.cc:288] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:37.412101 24358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:37.412209 24358 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [term 1 LEADER]: Becoming Leader. State: Replica: f7937ed4e69f4c0ca16b38d458fdaa33, State: Running, Role: LEADER
I20260619 08:18:37.412317 24358 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } }
I20260619 08:18:37.412322 24353 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:37.412639 24360 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7937ed4e69f4c0ca16b38d458fdaa33. Latest consensus state: current_term: 1 leader_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } } }
I20260619 08:18:37.412750 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:37.412920 24360 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:37.412830 24359 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7937ed4e69f4c0ca16b38d458fdaa33" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 39963 } } }
I20260619 08:18:37.413066 24359 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:37.413360 24363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:37.413865 24363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:37.416091 24363 catalog_manager.cc:1347] Generated new cluster ID: cfc3b9c9d13446fabaf08a6fc911bce3
I20260619 08:18:37.416163 24363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:37.416461 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:37.414731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37739 (local address 127.2.119.254:39963)
0619 08:18:37.414762 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:37.414763 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:37.415333 (+ 570us) server_negotiation.cc:407] Connection header received
0619 08:18:37.415394 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:37.415395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:37.415399 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:37.415468 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:37.416164 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:37.416165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:37.416165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:37.416169 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:37.416184 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:37.416188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:37.416256 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:37.416394 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 08:18:37.416406 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 08:18:37.447139 24363 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:37.447345 24363 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:37.451963 24363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7937ed4e69f4c0ca16b38d458fdaa33: Generated new TSK 0
I20260619 08:18:37.452152 24363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:37.461061 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24285
2026-06-19T08:18:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:37 dist-test-slave-d0b5 krb5kdc[24384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:37 dist-test-slave-d0b5 krb5kdc[24384](info): set up 2 sockets
Jun 19 08:18:37 dist-test-slave-d0b5 krb5kdc[24384](info): commencing operation
krb5kdc: starting...
W20260619 08:18:38.502512 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:38.632452 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45363
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:34143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:38.719192 24397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:38.719396 24397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:38.719455 24397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:38.721617 24397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:38.721688 24397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:38.721714 24397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:38.721764 24397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:38.721792 24397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:38.724339 24397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:34143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:38.724608 24397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:38.724858 24397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:38.730656 24403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:38.730691 24405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:38.730656 24402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:38.730731 24397 server_base.cc:1034] running on GCE node
I20260619 08:18:38.731078 24397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:38.731391 24397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:38.732550 24397 hybrid_clock.cc:648] HybridClock initialized: now 1781857118732535 us; error 35 us; skew 500 ppm
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857118, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:38.736032 24397 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:38.736686 24397 webserver.cc:460] Webserver started at http://127.2.119.254:37067/ using document root <none> and password file <none>
I20260619 08:18:38.736955 24397 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:38.737015 24397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:38.737150 24397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:38.738637 24397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "d56862ee18f746398d066b70f953e349"
format_stamp: "Formatted at 2026-06-19 08:18:38 on dist-test-slave-d0b5"
server_key: "0b18e3a0400dfa76584ae168eb1ed726"
server_key_iv: "93ff45310a12688b429967817944908b"
server_key_version: "encryptionkey@0"
I20260619 08:18:38.739033 24397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "d56862ee18f746398d066b70f953e349"
format_stamp: "Formatted at 2026-06-19 08:18:38 on dist-test-slave-d0b5"
server_key: "0b18e3a0400dfa76584ae168eb1ed726"
server_key_iv: "93ff45310a12688b429967817944908b"
server_key_version: "encryptionkey@0"
I20260619 08:18:38.740696 24397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:38.742592 24412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:38.742795 24397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 08:18:38.742897 24397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "d56862ee18f746398d066b70f953e349"
format_stamp: "Formatted at 2026-06-19 08:18:38 on dist-test-slave-d0b5"
server_key: "0b18e3a0400dfa76584ae168eb1ed726"
server_key_iv: "93ff45310a12688b429967817944908b"
server_key_version: "encryptionkey@0"
I20260619 08:18:38.742971 24397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:38.761407 24397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:38.763875 24397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:38.764022 24397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:38.768217 24397 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45363
I20260619 08:18:38.768253 24464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45363 every 8 connection(s)
I20260619 08:18:38.768611 24397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:38.769568 24465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:38.771910 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349: Bootstrap starting.
I20260619 08:18:38.772634 24465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:38.772971 24465 log.cc:824] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:38.773828 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349: No bootstrap required, opened a new log
I20260619 08:18:38.775432 24465 raft_consensus.cc:348] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } }
I20260619 08:18:38.775571 24465 raft_consensus.cc:374] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:38.775594 24465 raft_consensus.cc:729] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56862ee18f746398d066b70f953e349, State: Initialized, Role: FOLLOWER
I20260619 08:18:38.775764 24465 consensus_queue.cc:260] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } }
I20260619 08:18:38.775831 24465 raft_consensus.cc:388] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:38.775852 24465 raft_consensus.cc:482] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:38.775879 24465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:38.776784 24465 raft_consensus.cc:504] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } }
I20260619 08:18:38.776872 24465 leader_election.cc:302] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d56862ee18f746398d066b70f953e349; no voters:
I20260619 08:18:38.777079 24465 leader_election.cc:288] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:38.777192 24468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:38.777278 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24397
I20260619 08:18:38.777451 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:38.777410 24468 raft_consensus.cc:686] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [term 1 LEADER]: Becoming Leader. State: Replica: d56862ee18f746398d066b70f953e349, State: Running, Role: LEADER
I20260619 08:18:38.777532 24468 consensus_queue.cc:237] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } }
I20260619 08:18:38.777652 2527 external_mini_cluster.cc:1428] Setting key 2132c98a6a27d05c7260cb42c134fd0c
I20260619 08:18:38.777400 24465 sys_catalog.cc:564] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:38.778738 24469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d56862ee18f746398d066b70f953e349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } } }
I20260619 08:18:38.778836 24469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:38.779026 24470 sys_catalog.cc:455] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d56862ee18f746398d066b70f953e349. Latest consensus state: current_term: 1 leader_uuid: "d56862ee18f746398d066b70f953e349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d56862ee18f746398d066b70f953e349" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45363 } } }
I20260619 08:18:38.779093 24470 sys_catalog.cc:458] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:38.779246 24480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:38.779832 24480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:38.785579 24480 catalog_manager.cc:1347] Generated new cluster ID: db2ff98ca96849cf8027c62f556ed3eb
I20260619 08:18:38.785660 24480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:38.789173 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:38.778289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57795 (local address 127.2.119.254:45363)
0619 08:18:38.778441 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:38.778449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:38.778482 (+ 33us) server_negotiation.cc:407] Connection header received
0619 08:18:38.778602 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:38.778609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:38.778726 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:38.778911 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:38.779756 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:38.781150 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:38.781955 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:38.782588 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:38.782705 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:38.785155 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:38.785159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:38.785160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:38.785172 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:38.787473 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:38.787871 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:38.787874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:38.787874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:38.788015 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:38.788224 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:38.788225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:38.788225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:38.788550 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:38.788709 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:38.788792 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 08:18:38.788949 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260619 08:18:38.790134 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:38.809378 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:38.801745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44531 (local address 127.2.119.254:45363)
0619 08:18:38.801792 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:38.801794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:38.801973 (+ 179us) server_negotiation.cc:407] Connection header received
0619 08:18:38.802040 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:38.802041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:38.802054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:38.802104 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:38.802824 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:38.803572 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:38.804687 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:38.804997 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:38.805059 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:38.806564 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:38.806566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:38.806566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:38.806573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:38.808160 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:38.808555 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:38.808556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:38.808556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:38.808612 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:38.808828 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:38.808829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:38.808829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:38.808951 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:38.809036 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:38.809176 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 08:18:38.809251 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:18:38.819198 24480 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:38.819968 24480 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:38.824561 24480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d56862ee18f746398d066b70f953e349: Generated new TSK 0
I20260619 08:18:38.824747 24480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:38.834609 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24397
2026-06-19T08:18:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24496](info): set up 2 sockets
Jun 19 08:18:38 dist-test-slave-d0b5 krb5kdc[24496](info): commencing operation
krb5kdc: starting...
W20260619 08:18:39.910502 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:39 dist-test-slave-d0b5 krb5kdc[24496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:40.032445 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45949
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:37859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:40.112860 24509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:40.113090 24509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:40.113128 24509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:40.114274 24509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:40.114341 24509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:40.114367 24509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:40.114389 24509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:40.114418 24509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:40.115861 24509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:37859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:45949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:40.116127 24509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:40.116348 24509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:40.121328 24514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:40.121277 24515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:40.121352 24517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:40.121560 24509 server_base.cc:1034] running on GCE node
I20260619 08:18:40.121815 24509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:40.122117 24509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:40.123281 24509 hybrid_clock.cc:648] HybridClock initialized: now 1781857120123266 us; error 28 us; skew 500 ppm
Jun 19 08:18:40 dist-test-slave-d0b5 krb5kdc[24496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857120, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:40.125933 24509 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:40.126407 24509 webserver.cc:460] Webserver started at http://127.2.119.254:42601/ using document root <none> and password file <none>
I20260619 08:18:40.126647 24509 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:40.126699 24509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:40.126863 24509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:40.127887 24509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "5dc6a7d19e9d49eaade621ff7f6005b5"
format_stamp: "Formatted at 2026-06-19 08:18:40 on dist-test-slave-d0b5"
server_key: "ec00326cfad030eab32faa65517b54a7"
server_key_iv: "0b0123c1d1f0f56e597d18170badc948"
server_key_version: "encryptionkey@0"
I20260619 08:18:40.128268 24509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "5dc6a7d19e9d49eaade621ff7f6005b5"
format_stamp: "Formatted at 2026-06-19 08:18:40 on dist-test-slave-d0b5"
server_key: "ec00326cfad030eab32faa65517b54a7"
server_key_iv: "0b0123c1d1f0f56e597d18170badc948"
server_key_version: "encryptionkey@0"
I20260619 08:18:40.129757 24509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:40.130558 24524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:40.130764 24509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:40.130849 24509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "5dc6a7d19e9d49eaade621ff7f6005b5"
format_stamp: "Formatted at 2026-06-19 08:18:40 on dist-test-slave-d0b5"
server_key: "ec00326cfad030eab32faa65517b54a7"
server_key_iv: "0b0123c1d1f0f56e597d18170badc948"
server_key_version: "encryptionkey@0"
I20260619 08:18:40.130903 24509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:40.151024 24509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:40.153347 24509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:40.153460 24509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:40.157579 24509 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:45949
I20260619 08:18:40.157632 24576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:45949 every 8 connection(s)
I20260619 08:18:40.157996 24509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:40.158919 24577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:40.161204 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5: Bootstrap starting.
I20260619 08:18:40.161796 24577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:40.162084 24577 log.cc:824] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:40.162840 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5: No bootstrap required, opened a new log
I20260619 08:18:40.164399 24577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } }
I20260619 08:18:40.164556 24577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:40.164600 24577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc6a7d19e9d49eaade621ff7f6005b5, State: Initialized, Role: FOLLOWER
I20260619 08:18:40.164769 24577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } }
I20260619 08:18:40.164857 24577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:40.164903 24577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:40.164952 24577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:40.165795 24577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } }
I20260619 08:18:40.165915 24577 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc6a7d19e9d49eaade621ff7f6005b5; no voters:
I20260619 08:18:40.166095 24577 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:40.166148 24580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:40.166345 24577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:40.166350 24580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc6a7d19e9d49eaade621ff7f6005b5, State: Running, Role: LEADER
I20260619 08:18:40.166462 24580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } }
I20260619 08:18:40.166851 24582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc6a7d19e9d49eaade621ff7f6005b5. Latest consensus state: current_term: 1 leader_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } } }
I20260619 08:18:40.166947 24582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:40.167332 24581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc6a7d19e9d49eaade621ff7f6005b5" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 45949 } } }
I20260619 08:18:40.167402 24589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:40.167424 24581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:40.167649 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24509
I20260619 08:18:40.167765 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:40.167917 24589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:40.167940 2527 external_mini_cluster.cc:1428] Setting key c62a1846d0fa1ac09905804f7b517e8d
I20260619 08:18:40.169621 24589 catalog_manager.cc:1347] Generated new cluster ID: ae7184e4539048fcad4e80d9d68c4eb5
I20260619 08:18:40.169682 24589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:40.174032 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:40.168479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52665 (local address 127.2.119.254:45949)
0619 08:18:40.168755 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:40.168764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:40.168794 (+ 30us) server_negotiation.cc:407] Connection header received
0619 08:18:40.168899 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:40.168904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:40.168944 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:40.169088 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:40.169389 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:40.170758 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:40.171901 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:40.172503 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:40.172605 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:40.173352 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:40.173355 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:40.173356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:40.173549 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:40.173584 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:40.173597 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:40.173656 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:40.173760 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 08:18:40.173822 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":74,"threads_started":1}
I20260619 08:18:40.174898 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:40.179843 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:40.176518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38091 (local address 127.2.119.254:45949)
0619 08:18:40.176549 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:40.176551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:40.176558 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:40.176624 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:40.176624 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:40.176633 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:40.176683 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:40.176809 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:40.177512 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:40.178557 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:40.178855 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:40.178915 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:40.179615 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:40.179616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:40.179616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:40.179621 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:40.179634 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:40.179638 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:40.179686 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:40.179748 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:18:40.179767 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:18:40.194063 24589 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:40.194828 24589 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:40.200310 24589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc6a7d19e9d49eaade621ff7f6005b5: Generated new TSK 0
I20260619 08:18:40.200495 24589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:40.203039 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24509
2026-06-19T08:18:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:40 dist-test-slave-d0b5 krb5kdc[24608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:40 dist-test-slave-d0b5 krb5kdc[24608](info): set up 2 sockets
Jun 19 08:18:40 dist-test-slave-d0b5 krb5kdc[24608](info): commencing operation
krb5kdc: starting...
W20260619 08:18:41.242547 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:41 dist-test-slave-d0b5 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:41.359480 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41015
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:36965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:41.439352 24621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:41.439587 24621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:41.439627 24621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:41.440784 24621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:41.440852 24621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:41.440878 24621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:41.440910 24621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:41.440941 24621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:41.442426 24621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:36965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:41.442723 24621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:41.442956 24621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:41.447873 24626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:41.447948 24627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:41.447901 24629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:41.447986 24621 server_base.cc:1034] running on GCE node
I20260619 08:18:41.448385 24621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:41.448632 24621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:41.449776 24621 hybrid_clock.cc:648] HybridClock initialized: now 1781857121449761 us; error 31 us; skew 500 ppm
Jun 19 08:18:41 dist-test-slave-d0b5 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857121, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:41.452539 24621 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:41.453001 24621 webserver.cc:460] Webserver started at http://127.2.119.254:40427/ using document root <none> and password file <none>
I20260619 08:18:41.453243 24621 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:41.453338 24621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:41.453486 24621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:41.454547 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "6cf9be081cba469ca6b907b7cbc60415"
format_stamp: "Formatted at 2026-06-19 08:18:41 on dist-test-slave-d0b5"
server_key: "04429cb834603c85e006844cfd0a1840"
server_key_iv: "852ef34737b4ecbb72e9d42d1481ed2d"
server_key_version: "encryptionkey@0"
I20260619 08:18:41.454864 24621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "6cf9be081cba469ca6b907b7cbc60415"
format_stamp: "Formatted at 2026-06-19 08:18:41 on dist-test-slave-d0b5"
server_key: "04429cb834603c85e006844cfd0a1840"
server_key_iv: "852ef34737b4ecbb72e9d42d1481ed2d"
server_key_version: "encryptionkey@0"
I20260619 08:18:41.456302 24621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:41.457050 24636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:41.457250 24621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:41.457418 24621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "6cf9be081cba469ca6b907b7cbc60415"
format_stamp: "Formatted at 2026-06-19 08:18:41 on dist-test-slave-d0b5"
server_key: "04429cb834603c85e006844cfd0a1840"
server_key_iv: "852ef34737b4ecbb72e9d42d1481ed2d"
server_key_version: "encryptionkey@0"
I20260619 08:18:41.457507 24621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:41.461011 24621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:41.463568 24621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:41.463734 24621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:41.467859 24621 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41015
I20260619 08:18:41.467906 24688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41015 every 8 connection(s)
I20260619 08:18:41.468242 24621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:41.469211 24689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:41.471488 24689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415: Bootstrap starting.
I20260619 08:18:41.472020 24689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:41.472254 24689 log.cc:824] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:41.472940 24689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415: No bootstrap required, opened a new log
I20260619 08:18:41.473733 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24621
I20260619 08:18:41.473855 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:41.474025 2527 external_mini_cluster.cc:1428] Setting key 2e68b6921e4a16afca2cae66d720326a
I20260619 08:18:41.475149 24689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } }
I20260619 08:18:41.475328 24689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 08:18:41.475368 24689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cf9be081cba469ca6b907b7cbc60415, State: Initialized, Role: FOLLOWER
I20260619 08:18:41.475522 24689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } }
I20260619 08:18:41.475626 24689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 08:18:41.475687 24689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 08:18:41.475739 24689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:41.476517 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:41.474553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53321 (local address 127.2.119.254:41015)
0619 08:18:41.474763 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:41.474772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:41.474804 (+ 32us) server_negotiation.cc:407] Connection header received
0619 08:18:41.475103 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:41.475111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:41.475148 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:41.475297 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:41.476022 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:41.476025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:41.476026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:41.476029 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:41.476066 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:41.476077 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:41.476158 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:41.476237 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 08:18:41.476267 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260619 08:18:41.476758 24689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } }
I20260619 08:18:41.476862 24689 leader_election.cc:302] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cf9be081cba469ca6b907b7cbc60415; no voters:
I20260619 08:18:41.477005 24689 leader_election.cc:288] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 08:18:41.477026 24694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:41.477138 24694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [term 1 LEADER]: Becoming Leader. State: Replica: 6cf9be081cba469ca6b907b7cbc60415, State: Running, Role: LEADER
I20260619 08:18:41.477206 24694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } }
I20260619 08:18:41.477262 24689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:41.477487 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:41.477558 24694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cf9be081cba469ca6b907b7cbc60415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } } }
I20260619 08:18:41.477618 24694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:41.477746 24694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cf9be081cba469ca6b907b7cbc60415. Latest consensus state: current_term: 1 leader_uuid: "6cf9be081cba469ca6b907b7cbc60415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cf9be081cba469ca6b907b7cbc60415" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41015 } } }
I20260619 08:18:41.477802 24694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:41.478086 24699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:41.478590 24699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:41.480405 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:41.479255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39549 (local address 127.2.119.254:41015)
0619 08:18:41.479289 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:41.479291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:41.479430 (+ 139us) server_negotiation.cc:407] Connection header received
0619 08:18:41.479502 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:41.479503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:41.479507 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:41.479560 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:41.480177 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:41.480178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:41.480179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 08:18:41.480183 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:41.480197 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 08:18:41.480201 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:41.480267 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:41.480329 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 08:18:41.480342 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:41.480504 24699 catalog_manager.cc:1347] Generated new cluster ID: 37e0075de146448a8557ff7d06b19d53
I20260619 08:18:41.480547 24699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:41.488329 24699 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:41.488502 24699 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:41.497768 24699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cf9be081cba469ca6b907b7cbc60415: Generated new TSK 0
I20260619 08:18:41.497954 24699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:41.502141 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24621
2026-06-19T08:18:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1299 ms)
[----------] 18 tests from AuthTokenIssuingTest (24125 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:41 dist-test-slave-d0b5 krb5kdc[24720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:41 dist-test-slave-d0b5 krb5kdc[24720](info): set up 2 sockets
Jun 19 08:18:41 dist-test-slave-d0b5 krb5kdc[24720](info): commencing operation
krb5kdc: starting...
W20260619 08:18:42.530617 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:42.656749 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36231
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:42.744918 24733 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:42.745208 24733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:42.745260 24733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:42.745332 24733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:42.746608 24733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:42.746686 24733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:42.746714 24733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:42.746742 24733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:42.746770 24733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:42.748287 24733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:36231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:42.748571 24733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:42.748816 24733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:42.753979 24738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:42.754067 24741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:42.754103 24733 server_base.cc:1034] running on GCE node
W20260619 08:18:42.753979 24739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:42.754493 24733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:42.754755 24733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:42.755903 24733 hybrid_clock.cc:648] HybridClock initialized: now 1781857122755883 us; error 45 us; skew 500 ppm
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:42.758402 24733 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:42.758894 24733 webserver.cc:460] Webserver started at http://127.2.119.254:42653/ using document root <none> and password file <none>
I20260619 08:18:42.759135 24733 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:42.759179 24733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:42.759349 24733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:42.760437 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "9cf71efb47644f7483f4b527c7b89513"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "be98a43f372511e9aee2b8746f7b4326"
server_key_iv: "a8063462f5e127a1dd173c784663e2bd"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.760815 24733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "9cf71efb47644f7483f4b527c7b89513"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "be98a43f372511e9aee2b8746f7b4326"
server_key_iv: "a8063462f5e127a1dd173c784663e2bd"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.762231 24733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 08:18:42.762928 24748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:42.763119 24733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:42.763208 24733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "9cf71efb47644f7483f4b527c7b89513"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "be98a43f372511e9aee2b8746f7b4326"
server_key_iv: "a8063462f5e127a1dd173c784663e2bd"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.763257 24733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:42.774828 24733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:42.777084 24733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:42.777199 24733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:42.780910 24733 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:36231
I20260619 08:18:42.780961 24800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:36231 every 8 connection(s)
I20260619 08:18:42.781289 24733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:42.781723 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24733
I20260619 08:18:42.781843 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:42.782048 2527 external_mini_cluster.cc:1428] Setting key 94b28e151d0f3bc384c8925e4551690c
I20260619 08:18:42.782634 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:42.784538 24801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.119.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.253@KRBTEST.COM" created.
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:42.797996 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.785407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36231 (local address 127.0.0.1:54222)
0619 08:18:42.785697 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.785724 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:42.786173 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:42.786521 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:42.786523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:42.786768 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:42.786991 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.786995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.788625 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.788626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.789441 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.789443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.790182 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.790183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.790242 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.790721 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:42.790730 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:42.793666 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:42.796084 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.796091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.796094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.796394 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.796732 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.796733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.796734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.796856 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.797236 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:42.797244 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:42.797355 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 08:18:42.797461 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 08:18:42.797784 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":141,"thread_start_us":142,"threads_started":1}
I20260619 08:18:42.798005 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.785410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.2.119.254:36231)
0619 08:18:42.785624 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:42.785634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:42.785823 (+ 189us) server_negotiation.cc:407] Connection header received
0619 08:18:42.786336 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:42.786339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:42.786364 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:42.786513 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:42.787098 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.788508 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.789523 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.790038 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.790167 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.793763 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:42.793771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:42.793771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:42.793779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:42.795997 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.796527 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.796530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.796531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.796643 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.796936 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.796937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.796937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.797154 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:42.797219 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:42.797532 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 08:18:42.797692 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260619 08:18:42.799647 24801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:42.800043 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.799901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49478)
0619 08:18:42.799957 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.800009 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 08:18:42.800256 24751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
I20260619 08:18:42.801385 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.801226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49482)
0619 08:18:42.801326 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.801357 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 08:18:42.801496 24801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:35623: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
I20260619 08:18:42.811901 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:35623
--webserver_interface=127.2.119.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:18:42.850749 24801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } attempt: 1
I20260619 08:18:42.851302 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.851147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49496)
0619 08:18:42.851217 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.851267 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:18:42.851693 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.851590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49510)
0619 08:18:42.851639 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.851668 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:18:42.851825 24801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:35623: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
W20260619 08:18:42.894438 24808 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:42.894687 24808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:42.894729 24808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:42.894773 24808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:42.895915 24808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:42.895983 24808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:42.896008 24808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:42.896036 24808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:42.896067 24808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:42.897548 24808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:35623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.119.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:42.897841 24808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:42.898069 24808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:42.903029 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:42.903064 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:42.903083 24808 server_base.cc:1034] running on GCE node
W20260619 08:18:42.903050 24816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:42.903550 24808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:42.903848 24808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:42.905009 24808 hybrid_clock.cc:648] HybridClock initialized: now 1781857122904984 us; error 37 us; skew 500 ppm
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:42.907795 24808 init.cc:377] Logged in from keytab as kudu/127.2.119.253@KRBTEST.COM (short username kudu)
I20260619 08:18:42.908216 24808 webserver.cc:460] Webserver started at http://127.2.119.253:42845/ using document root <none> and password file <none>
I20260619 08:18:42.908450 24808 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:42.908519 24808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:42.908669 24808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:42.909778 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data/instance:
uuid: "1e3d56bf4152402babad50dbcde2e3a9"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "84481f0a4539ece9269ed822d037cf0f"
server_key_iv: "a31f510c78019d92f432003f8519a39b"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.910136 24808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal/instance:
uuid: "1e3d56bf4152402babad50dbcde2e3a9"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "84481f0a4539ece9269ed822d037cf0f"
server_key_iv: "a31f510c78019d92f432003f8519a39b"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.911458 24808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 08:18:42.912250 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:42.912443 24808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 08:18:42.912545 24808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "1e3d56bf4152402babad50dbcde2e3a9"
format_stamp: "Formatted at 2026-06-19 08:18:42 on dist-test-slave-d0b5"
server_key: "84481f0a4539ece9269ed822d037cf0f"
server_key_iv: "a31f510c78019d92f432003f8519a39b"
server_key_version: "encryptionkey@0"
I20260619 08:18:42.912618 24808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:42.919972 24801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } attempt: 2
I20260619 08:18:42.920403 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.920279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49524)
0619 08:18:42.920336 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.920374 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:18:42.920723 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.920643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49528)
0619 08:18:42.920683 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.920707 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:18:42.920837 24801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:35623: Network error: Client connection negotiation failed: client connection to 127.2.119.253:35623: connect: Connection refused (error 111)
I20260619 08:18:42.922847 24808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:42.924805 24808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:42.924957 24808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:42.929158 24808 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.253:35623
I20260619 08:18:42.929261 24875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.253:35623 every 8 connection(s)
I20260619 08:18:42.929628 24808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
I20260619 08:18:42.930624 24876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:42.932258 24876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:42.936836 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24808
I20260619 08:18:42.936959 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-1/wal/instance
I20260619 08:18:42.937152 2527 external_mini_cluster.cc:1428] Setting key ae6235206f13c6c30cb4f208fa1de525
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:42.942065 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.932949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.2.119.254:36231)
0619 08:18:42.933006 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:42.933008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:42.933233 (+ 225us) server_negotiation.cc:407] Connection header received
0619 08:18:42.933679 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:42.933680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:42.933691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:42.933750 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:42.934392 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.934886 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.936269 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.936476 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.936537 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.938912 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:42.938914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:42.938914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:42.938920 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:42.940399 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.940957 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.940959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.940959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.941023 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.941364 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.941365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.941366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.941526 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:42.941591 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:42.941884 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 08:18:42.941974 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 08:18:42.942248 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.932880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36231 (local address 127.0.0.1:54238)
0619 08:18:42.933035 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.933077 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:42.933590 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:42.933809 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:42.933813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:42.934070 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:42.934262 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.934266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.934974 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.934974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.935811 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.935814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.936567 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.936568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.936675 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.937294 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:42.937326 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:42.938812 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:42.940512 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.940520 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.940522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.940878 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.941091 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.941092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.941093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.941250 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.941593 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:42.941601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:42.941739 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 08:18:42.941842 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 08:18:42.942000 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260619 08:18:42.942768 24876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.119.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.252@KRBTEST.COM" created.
Jun 19 08:18:42 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:42.951638 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.943045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49538)
0619 08:18:42.943266 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.943279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:42.943345 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:42.943500 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:42.943501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:42.943663 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:42.943733 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.943735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.944610 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.944611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.945474 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:42.945476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.945923 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.945924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:42.945978 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.946352 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:42.946356 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:42.948720 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:42.950621 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.950622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.950622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.950785 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.951002 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:42.951002 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:42.951003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:42.951056 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:42.951338 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:42.951339 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:42.951376 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 08:18:42.951437 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 08:18:42.951533 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260619 08:18:42.951736 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.943122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.2.119.253:35623)
0619 08:18:42.943268 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:42.943271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:42.943329 (+ 58us) server_negotiation.cc:407] Connection header received
0619 08:18:42.943403 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:42.943404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:42.943415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:42.943474 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:42.943835 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.944517 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.945545 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:42.945854 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:42.945914 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:42.948822 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:42.948830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:42.948830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:42.948836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:42.950526 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.950880 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.950882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.950882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.950936 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:42.951127 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:42.951127 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:42.951128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:42.951262 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:42.951348 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:42.951465 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 08:18:42.951602 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260619 08:18:42.952473 24876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:42.952848 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.952712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39708)
0619 08:18:42.952781 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.952828 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 08:18:42.953032 24825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
I20260619 08:18:42.953454 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:42.953376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39714)
0619 08:18:42.953413 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:42.953437 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 08:18:42.953579 24876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:43303: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
I20260619 08:18:42.965135 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:43303
--webserver_interface=127.2.119.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:45971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:18:43.002831 24876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } attempt: 1
I20260619 08:18:43.003417 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.003261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39726)
0619 08:18:43.003342 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.003385 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 08:18:43.003877 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.003747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39734)
0619 08:18:43.003820 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.003853 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 08:18:43.004009 24876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:43303: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
I20260619 08:18:43.012019 24801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } attempt: 3
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:43.019649 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.012433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.2.119.253:35623)
0619 08:18:43.012494 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.012496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.012516 (+ 20us) server_negotiation.cc:407] Connection header received
0619 08:18:43.012548 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.012549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.012558 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.012616 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.013003 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.013611 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.014606 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.014861 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.014918 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.016964 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.016965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.016966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.016973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.018463 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.018845 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.018847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.018848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.018910 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.019116 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.019116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.019117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.019236 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.019309 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.019472 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.019552 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 08:18:43.019663 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.012382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49554)
0619 08:18:43.012440 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.012451 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.012495 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.012641 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.012642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.012825 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.012902 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.012905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.013688 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.013690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.014518 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.014520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.014933 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.014934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.015010 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.015453 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.015459 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.016871 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.018581 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.018582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.018583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.018767 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.018981 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.018982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.018982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.019050 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.019313 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.019314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.019362 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.019448 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.019552 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:18:43.020406 24801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:43.020752 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.020643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39740)
0619 08:18:43.020692 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.020723 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 08:18:43.021236 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.021160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39742)
0619 08:18:43.021188 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.021215 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 08:18:43.021391 24801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:43303: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
W20260619 08:18:43.048938 24883 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:43.049149 24883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:43.049170 24883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:43.049191 24883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:43.050354 24883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:43.050391 24883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:43.050406 24883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:43.050419 24883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:43.050431 24883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:43.051901 24883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:45971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:36231,127.2.119.253:35623,127.2.119.252:43303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:43303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.119.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:43.052085 24883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:43.052255 24883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 08:18:43.052554 24801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } attempt: 1
I20260619 08:18:43.053117 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.052961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39754)
0619 08:18:43.053047 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.053088 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 08:18:43.053572 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.053460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39756)
0619 08:18:43.053511 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.053546 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:18:43.053722 24801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:43303: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
W20260619 08:18:43.057535 24891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:43.057555 24883 server_base.cc:1034] running on GCE node
W20260619 08:18:43.057581 24888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:43.057552 24889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:43.058012 24883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:43.058252 24883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:43.059396 24883 hybrid_clock.cc:648] HybridClock initialized: now 1781857123059374 us; error 36 us; skew 500 ppm
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857123, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:43.062011 24883 init.cc:377] Logged in from keytab as kudu/127.2.119.252@KRBTEST.COM (short username kudu)
I20260619 08:18:43.062482 24883 webserver.cc:460] Webserver started at http://127.2.119.252:46337/ using document root <none> and password file <none>
I20260619 08:18:43.062703 24883 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:43.062772 24883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:43.062924 24883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:43.063936 24883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data/instance:
uuid: "04b2927a745b4ade9cce8fb5aa37439f"
format_stamp: "Formatted at 2026-06-19 08:18:43 on dist-test-slave-d0b5"
server_key: "2f3b648cedf37b664b6fdb3d88d669d3"
server_key_iv: "8e886e81eadf0523d0512effae8c9828"
server_key_version: "encryptionkey@0"
I20260619 08:18:43.064296 24883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal/instance:
uuid: "04b2927a745b4ade9cce8fb5aa37439f"
format_stamp: "Formatted at 2026-06-19 08:18:43 on dist-test-slave-d0b5"
server_key: "2f3b648cedf37b664b6fdb3d88d669d3"
server_key_iv: "8e886e81eadf0523d0512effae8c9828"
server_key_version: "encryptionkey@0"
I20260619 08:18:43.065618 24883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 08:18:43.066552 24898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:43.066766 24883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:43.066903 24883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "04b2927a745b4ade9cce8fb5aa37439f"
format_stamp: "Formatted at 2026-06-19 08:18:43 on dist-test-slave-d0b5"
server_key: "2f3b648cedf37b664b6fdb3d88d669d3"
server_key_iv: "8e886e81eadf0523d0512effae8c9828"
server_key_version: "encryptionkey@0"
I20260619 08:18:43.066992 24883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:43.072170 24876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } attempt: 2
I20260619 08:18:43.072585 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.072453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39772)
0619 08:18:43.072514 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.072547 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 08:18:43.072996 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.072898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39782)
0619 08:18:43.072947 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.072974 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 08:18:43.073108 24876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:43303: Network error: Client connection negotiation failed: client connection to 127.2.119.252:43303: connect: Connection refused (error 111)
I20260619 08:18:43.088403 24883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:43.090953 24883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:43.091132 24883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:43.095387 24883 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.252:43303
I20260619 08:18:43.095427 24950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.252:43303 every 8 connection(s)
I20260619 08:18:43.095818 24883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
I20260619 08:18:43.096830 24951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:43.098477 24951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:43.100934 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 24883
I20260619 08:18:43.101011 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781856856097588-2527-0/minicluster-data/master-2/wal/instance
I20260619 08:18:43.101204 2527 external_mini_cluster.cc:1428] Setting key 05114ea6c7d9514c6145f117a2fc43f9
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857123, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:18:43.109287 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.099221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.2.119.254:36231)
0619 08:18:43.099325 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.099328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.099503 (+ 175us) server_negotiation.cc:407] Connection header received
0619 08:18:43.099998 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.100000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.100010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.100074 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.100801 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.101442 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.102703 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.102895 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.102952 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.105075 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.105078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.105078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.105086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.107604 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.108215 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.108216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.108217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.108283 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.108610 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.108611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.108611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.108765 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.108841 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.109087 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.109173 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 08:18:43.109444 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.099130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:36231 (local address 127.0.0.1:54250)
0619 08:18:43.099388 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.099418 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.099877 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.100229 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.100234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.100489 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.100682 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.100686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.101534 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.101536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.102580 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.102582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.102995 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.102996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.103129 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.103654 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.103663 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.104983 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.107705 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.107772 (+ 67us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.107775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.108129 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.108374 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.108374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.108375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.108538 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.108837 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.108842 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.108955 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.109038 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.109198 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":80,"threads_started":1}
I20260619 08:18:43.110047 24951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:43.110792 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.101692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.2.119.252:43303)
0619 08:18:43.101881 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.101886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.101908 (+ 22us) server_negotiation.cc:407] Connection header received
0619 08:18:43.101960 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.101963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.101982 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.102061 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.102637 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.103576 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.104383 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.104739 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.104824 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.107469 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.107478 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.107479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.107485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.109458 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.109885 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.109888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.109888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.109949 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.110167 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.110168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.110168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.110339 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.110462 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.110549 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.110677 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857123, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:43.117746 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.110307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:35623 (local address 127.0.0.1:49560)
0619 08:18:43.110361 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.110373 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.110416 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.110576 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.110577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.110834 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.110879 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.110882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.111661 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.111662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.112420 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.112422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.112868 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.112869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.112934 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.113283 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.113288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.114744 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.116395 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.116396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.116397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.116581 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.117033 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.117034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.117035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.117098 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.117471 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.117472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.117521 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.117600 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.117684 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:18:43.117923 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.110398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.2.119.253:35623)
0619 08:18:43.110466 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.110468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.110475 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:43.110498 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.110499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.110509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.110561 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.110953 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.111574 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.112493 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.112793 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.112870 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.114815 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.114816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.114817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.114825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.116282 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.116655 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.116656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.116657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.116712 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.117192 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.117193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.117193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.117394 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.117459 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.117717 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.117804 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 08:18:43.118371 24951 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:43.119585 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.111573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.2.119.254:36231)
0619 08:18:43.111608 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.111610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.111618 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:18:43.111662 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.111663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.111672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.111720 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.112251 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.112878 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.113903 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.114206 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.114275 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.116794 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.116796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.116796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.116803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.118544 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.118881 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.118882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.118882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.118944 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.119082 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.119083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.119083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.119213 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.119363 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.119468 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.119527 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857123, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:43.125720 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.118656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39790)
0619 08:18:43.118708 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.118734 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.118771 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.118919 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.118919 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.119063 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.119096 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.119097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.119959 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.119960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.120551 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.120553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.121051 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.121052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.121101 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.121485 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.121490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.122819 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.124505 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.124506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.124506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.124652 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.124897 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.124898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.124899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.124974 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.125306 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.125307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.125396 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.125486 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.125589 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:18:43.125770 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.118757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.2.119.252:43303)
0619 08:18:43.118796 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.118798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.118807 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:43.118825 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.118825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.118834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.118887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.119153 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.119878 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.120627 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.120973 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.121038 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.122902 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.122903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.122904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.122910 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.124424 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.124723 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.124724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.124724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.124779 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.125051 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.125052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.125052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.125183 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.125260 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.125511 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.125642 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":24}
I20260619 08:18:43.127285 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: Bootstrap starting.
I20260619 08:18:43.128034 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.120331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.2.119.253:35623)
0619 08:18:43.120365 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.120366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.120374 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:18:43.120393 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.120394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.120402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.120445 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.121050 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.121791 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.122735 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.123059 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.123137 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.125339 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.125340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.125340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.125347 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.126986 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.127323 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.127323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.127324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.127378 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.127561 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.127562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.127562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.127669 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.127838 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.127872 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.127938 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:43.128122 24951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:43.128372 24951 log.cc:824] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:43.128935 24801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } attempt: 2
I20260619 08:18:43.129252 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: No bootstrap required, opened a new log
I20260619 08:18:43.129753 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:43.131232 24951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.131404 24951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b2927a745b4ade9cce8fb5aa37439f, State: Initialized, Role: FOLLOWER
I20260619 08:18:43.130721 24775 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9cf71efb47644f7483f4b527c7b89513"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54254
I20260619 08:18:43.131512 24951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.131844 24951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:43.131799 24957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.131916 24957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
W20260619 08:18:43.133781 24968 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:43.133833 24968 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:43.136963 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.129177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39798)
0619 08:18:43.129231 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.129242 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.129278 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.129528 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.129529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.129681 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.129723 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.129725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.130541 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.130542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.131383 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.131386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.131857 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.131858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.131920 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.132302 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.132308 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.133848 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.135849 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.135851 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.135851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.136067 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.136267 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.136267 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.136268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.136322 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.136649 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.136650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.136699 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.136763 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.136862 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:18:43.137260 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.129212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.2.119.252:43303)
0619 08:18:43.129411 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.129413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.129420 (+ 7us) server_negotiation.cc:407] Connection header received
0619 08:18:43.129440 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.129440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.129449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.129503 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.129795 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.130446 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.131487 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.131766 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.132091 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.133937 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.133938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.133938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.133945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.135743 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.136147 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.136148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.136148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.136200 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.136401 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.136402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.136402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.136560 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.136619 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.137105 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.137179 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260619 08:18:43.138608 24801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: Bootstrap starting.
I20260619 08:18:43.139228 24801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:43.139510 24801 log.cc:824] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:43.134230 24883 master_runner.cc:186] Error getting master registration for 127.2.119.254:36231: OK, instance_id { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" instance_seqno: 1781857122779915 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:18:43.139900 24883 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:18:43.140355 24801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: No bootstrap required, opened a new log
I20260619 08:18:43.141734 24801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.141872 24801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cf71efb47644f7483f4b527c7b89513, State: Initialized, Role: FOLLOWER
I20260619 08:18:43.141948 24801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.142261 24801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:43.142217 24971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.142287 24775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:18:43.142314 24971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.142344 24775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:43.143217 24775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.143496 24775 leader_election.cc:288] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [CANDIDATE]: Term 1 election: Requested vote from peers 1e3d56bf4152402babad50dbcde2e3a9 (127.2.119.253:35623), 04b2927a745b4ade9cce8fb5aa37439f (127.2.119.252:43303)
I20260619 08:18:43.144021 24925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9cf71efb47644f7483f4b527c7b89513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04b2927a745b4ade9cce8fb5aa37439f"
I20260619 08:18:43.144126 24925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:43.144977 24925 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cf71efb47644f7483f4b527c7b89513 in term 1.
I20260619 08:18:43.145193 24750 leader_election.cc:302] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04b2927a745b4ade9cce8fb5aa37439f, 9cf71efb47644f7483f4b527c7b89513; no voters:
I20260619 08:18:43.145679 24971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:43.145828 24971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [term 1 LEADER]: Becoming Leader. State: Replica: 9cf71efb47644f7483f4b527c7b89513, State: Running, Role: LEADER
I20260619 08:18:43.145910 24971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.148073 24733 master_runner.cc:186] Error getting master registration for 127.2.119.253:35623: OK, instance_id { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" instance_seqno: 1781857122928060 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:18:43.150270 24733 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:18:43.150513 24990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cf71efb47644f7483f4b527c7b89513. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.150594 24990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:43.151161 24982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:43.151211 24982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:43.154224 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.145153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.2.119.252:43303)
0619 08:18:43.145199 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.145200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.146516 (+ 1316us) server_negotiation.cc:407] Connection header received
0619 08:18:43.146648 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.146649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.146659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.146688 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.147410 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.148118 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.149102 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.149417 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.149460 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.151120 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.151122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.151122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.151130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.152795 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.153261 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.153263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.153263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.153351 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.153576 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.153577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.153578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.153720 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.153810 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.154054 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.154126 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:18:43.154935 24992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:43.155112 24992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:43.156670 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.144998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.2.119.253:35623)
0619 08:18:43.145060 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.145062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.146422 (+ 1360us) server_negotiation.cc:407] Connection header received
0619 08:18:43.146623 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.146624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.146635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.146687 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.147753 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.148444 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.149811 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.150145 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.150192 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.152707 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.152708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.152708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.152717 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.155393 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.155802 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.155803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.155803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.155863 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.156058 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.156059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.156060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.156188 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.156269 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.156461 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.156551 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:18:43.157467 24804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.144660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.2.119.254:36231)
0619 08:18:43.144697 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.144699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.145445 (+ 746us) server_negotiation.cc:407] Connection header received
0619 08:18:43.145491 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.145492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.145501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.145530 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.146319 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.147062 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.152206 (+ 5144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.152504 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.152545 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.154148 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.154149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.154149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.154157 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.155948 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.156350 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.156351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.156351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.156429 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.156662 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.156663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.156664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.156800 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.156887 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.157279 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.157380 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:43.162277 24925 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [term 1 FOLLOWER]: Refusing update from remote peer 9cf71efb47644f7483f4b527c7b89513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:43.162591 24990 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:43.163861 24957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cf71efb47644f7483f4b527c7b89513. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.163946 24957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.164233 24876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } attempt: 3
I20260619 08:18:43.165567 24990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.165666 24990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:43.165980 24957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.166069 24957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.167062 24995 mvcc.cc:205] Tried to move back new op lower bound from 7298486776469958656 to 7298486776406347776. Current Snapshot: MvccSnapshot[applied={T|T < 7298486776469958656}]
Jun 19 08:18:43 dist-test-slave-d0b5 krb5kdc[24720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857122, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:18:43.169775 24992 catalog_manager.cc:1347] Generated new cluster ID: 0dc2ade5eb394302b6ca2c0630228606
I20260619 08:18:43.169827 24992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:43.173732 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.164467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:43303 (local address 127.0.0.1:39810)
0619 08:18:43.164510 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:43.164520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:43.164553 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:43.164693 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:43.164694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:43.164847 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:43.164892 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.164894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.165846 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.165847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.166640 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:43.166644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.167363 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.167364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:43.167424 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.168082 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:43.168090 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:43.169733 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:43.172609 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.172611 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.172612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.172825 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.173027 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:43.173028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:43.173028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:43.173083 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:43.173419 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:43.173420 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:43.173466 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 08:18:43.173529 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 08:18:43.173630 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 08:18:43.174299 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:43.164520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.2.119.252:43303)
0619 08:18:43.164578 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.164580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.164588 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:18:43.164620 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.164621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.164631 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.164692 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.164959 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.165726 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.166880 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:43.167166 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:43.167228 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:43.169830 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:43.169831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:43.169832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:43.169839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:43.172518 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.172907 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.172908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.172908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.172963 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:43.173144 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:43.173145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:43.173146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:43.173269 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:43.173399 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:43.174136 (+ 737us) server_negotiation.cc:299] Negotiation successful
0619 08:18:43.174212 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 08:18:43.175925 24876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: Bootstrap starting.
I20260619 08:18:43.176635 24876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:43.176905 24876 log.cc:824] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:43.177932 24876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: No bootstrap required, opened a new log
I20260619 08:18:43.179282 24992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:43.179967 24992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:43.179962 24876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.180089 24876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3d56bf4152402babad50dbcde2e3a9, State: Initialized, Role: FOLLOWER
I20260619 08:18:43.180163 24876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } }
I20260619 08:18:43.180464 24876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:43.180487 24850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9cf71efb47644f7483f4b527c7b89513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e3d56bf4152402babad50dbcde2e3a9"
I20260619 08:18:43.180606 24850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:43.181905 24850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cf71efb47644f7483f4b527c7b89513 in term 1.
I20260619 08:18:43.182508 24849 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [term 1 FOLLOWER]: Refusing update from remote peer 9cf71efb47644f7483f4b527c7b89513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:43.183084 24990 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:43.184602 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cf71efb47644f7483f4b527c7b89513. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.184738 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.184824 25016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.184902 25016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.185097 25001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.185192 25001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:43.186467 25014 mvcc.cc:205] Tried to move back new op lower bound from 7298486776537743360 to 7298486776406347776. Current Snapshot: MvccSnapshot[applied={T|T < 7298486776469958656}]
W20260619 08:18:43.186530 25012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:43.186583 25012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:43.194114 24990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cf71efb47644f7483f4b527c7b89513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cf71efb47644f7483f4b527c7b89513" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 36231 } } peers { permanent_uuid: "1e3d56bf4152402babad50dbcde2e3a9" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 35623 } } peers { permanent_uuid: "04b2927a745b4ade9cce8fb5aa37439f" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 43303 } } }
I20260619 08:18:43.194228 24990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513 [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:43.195338 24992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cf71efb47644f7483f4b527c7b89513: Generated new TSK 0
I20260619 08:18:43.195516 24992 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 08:18:43.220861 24955 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.219201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.2.119.252:43303)
0619 08:18:43.219246 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.219249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.220180 (+ 931us) server_negotiation.cc:407] Connection header received
0619 08:18:43.220287 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.220289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.220300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.220331 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.220752 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39812: BlockingRecv error: recv got EOF from 127.0.0.1:39812 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:43.221683 24804 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.219416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.2.119.254:36231)
0619 08:18:43.219455 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.219457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.221136 (+ 1679us) server_negotiation.cc:407] Connection header received
0619 08:18:43.221233 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.221234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.221244 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.221273 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.221619 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54278: BlockingRecv error: recv got EOF from 127.0.0.1:54278 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:43.222396 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.219903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.2.119.253:35623)
0619 08:18:43.219941 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.219943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.221908 (+ 1965us) server_negotiation.cc:407] Connection header received
0619 08:18:43.222001 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.222002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.222012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.222040 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.222335 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49592: BlockingRecv error: recv got EOF from 127.0.0.1:49592 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:18:43.224614 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.223900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.2.119.253:35623)
0619 08:18:43.223932 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.223934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.224129 (+ 195us) server_negotiation.cc:407] Connection header received
0619 08:18:43.224227 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.224228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.224245 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.224273 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.224571 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49604: BlockingRecv error: recv got EOF from 127.0.0.1:49604 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 08:18:43.227535 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.226821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.2.119.253:35623)
0619 08:18:43.226863 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.226865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.227089 (+ 224us) server_negotiation.cc:407] Connection header received
0619 08:18:43.227176 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.227177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.227186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.227213 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.227495 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49606: BlockingRecv error: recv got EOF from 127.0.0.1:49606 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 08:18:43.231298 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.230581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49622 (local address 127.2.119.253:35623)
0619 08:18:43.230612 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.230613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.230846 (+ 233us) server_negotiation.cc:407] Connection header received
0619 08:18:43.230938 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.230939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.230948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.230975 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.231257 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49622: BlockingRecv error: recv got EOF from 127.0.0.1:49622 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 08:18:43.237246 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.236485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.2.119.253:35623)
0619 08:18:43.236519 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.236521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.236749 (+ 228us) server_negotiation.cc:407] Connection header received
0619 08:18:43.236853 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.236854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.236864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.236896 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.237199 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49628: BlockingRecv error: recv got EOF from 127.0.0.1:49628 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:18:43.247320 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.246475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.2.119.253:35623)
0619 08:18:43.246511 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.246513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.246750 (+ 237us) server_negotiation.cc:407] Connection header received
0619 08:18:43.246861 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.246862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.246872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.246914 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.247255 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49638: BlockingRecv error: recv got EOF from 127.0.0.1:49638 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:18:43.265552 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.264592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.2.119.253:35623)
0619 08:18:43.264633 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.264636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.264934 (+ 298us) server_negotiation.cc:407] Connection header received
0619 08:18:43.265040 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.265042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.265054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.265084 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.265495 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49642: BlockingRecv error: recv got EOF from 127.0.0.1:49642 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 08:18:43.300020 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.298892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.2.119.253:35623)
0619 08:18:43.298941 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.298943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.299243 (+ 300us) server_negotiation.cc:407] Connection header received
0619 08:18:43.299355 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.299356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.299368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.299400 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.299957 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49648: BlockingRecv error: recv got EOF from 127.0.0.1:49648 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 08:18:43.366369 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.365515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49658 (local address 127.2.119.253:35623)
0619 08:18:43.365613 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.365616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.365727 (+ 111us) server_negotiation.cc:407] Connection header received
0619 08:18:43.365771 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.365773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.365782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.365834 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.366275 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49658: BlockingRecv error: recv got EOF from 127.0.0.1:49658 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 08:18:43.496855 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.496009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.2.119.253:35623)
0619 08:18:43.496138 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.496141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.496238 (+ 97us) server_negotiation.cc:407] Connection header received
0619 08:18:43.496330 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.496331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.496348 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.496427 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.496771 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49664: BlockingRecv error: recv got EOF from 127.0.0.1:49664 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260619 08:18:43.754979 24880 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:43.754322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.2.119.253:35623)
0619 08:18:43.754416 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:43.754418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:43.754426 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:18:43.754462 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:43.754463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:43.754472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:43.754539 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:43.754898 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49676: BlockingRecv error: recv got EOF from 127.0.0.1:49676 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 08:18:44.134267 24968 catalog_manager.cc:1259] Loaded cluster ID: 0dc2ade5eb394302b6ca2c0630228606
I20260619 08:18:44.134344 24968 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: loading cluster ID for follower catalog manager: success
I20260619 08:18:44.135210 24968 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: acquiring CA information for follower catalog manager: success
I20260619 08:18:44.135535 24968 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04b2927a745b4ade9cce8fb5aa37439f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:18:44.187019 25012 catalog_manager.cc:1259] Loaded cluster ID: 0dc2ade5eb394302b6ca2c0630228606
I20260619 08:18:44.187096 25012 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: loading cluster ID for follower catalog manager: success
I20260619 08:18:44.187937 25012 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: acquiring CA information for follower catalog manager: success
I20260619 08:18:44.188194 25012 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e3d56bf4152402babad50dbcde2e3a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:18:44.271046 25085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:44.268217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.2.119.253:35623)
0619 08:18:44.268357 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:44.268359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:44.268390 (+ 31us) server_negotiation.cc:407] Connection header received
0619 08:18:44.268489 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:44.268490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:44.268549 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:44.268636 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:44.268794 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.269412 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.270224 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.270437 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.270505 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:44.270680 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:44.270810 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:44.270897 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:44.270921 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 08:18:44.270943 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260619 08:18:44.275401 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:44.272974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.2.119.252:43303)
0619 08:18:44.273109 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:44.273111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:44.273120 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:44.273157 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:44.273157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:44.273192 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:44.273266 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:44.273404 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.273881 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.274549 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.274791 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.274847 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:44.275024 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:44.275122 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:44.275211 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:44.275247 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 08:18:44.275266 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260619 08:18:44.279838 25085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:44.277322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.2.119.253:35623)
0619 08:18:44.277361 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:44.277362 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:44.277517 (+ 155us) server_negotiation.cc:407] Connection header received
0619 08:18:44.277575 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:44.277576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:44.277587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:44.277634 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:44.277788 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.278246 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.279069 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.279354 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.279424 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:44.279564 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:44.279595 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:44.279647 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:44.279719 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 08:18:44.279739 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:44.280165 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:44.277119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.2.119.254:36231)
0619 08:18:44.277369 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:44.277372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:44.277380 (+ 8us) server_negotiation.cc:407] Connection header received
0619 08:18:44.277475 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:44.277477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:44.277529 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:44.277611 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:44.277711 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.278227 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.279261 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.279510 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.279568 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:44.279812 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:44.279922 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:44.279987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:44.280041 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:18:44.280066 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
I20260619 08:18:44.281080 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:44.277150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.2.119.252:43303)
0619 08:18:44.277289 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:44.277313 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:44.277471 (+ 158us) server_negotiation.cc:407] Connection header received
0619 08:18:44.277545 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:44.277546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:44.277559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:44.277608 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:44.278358 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.279112 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.280349 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:44.280608 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:44.280677 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:44.280845 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:44.280872 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:44.280912 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:44.280966 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 08:18:44.280983 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260619 08:18:44.282238 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24733
I20260619 08:18:44.287472 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24808
I20260619 08:18:44.292706 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 24883
2026-06-19T08:18:44Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2792 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 08:18:44 dist-test-slave-d0b5 krb5kdc[25102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 08:18:44 dist-test-slave-d0b5 krb5kdc[25102](info): set up 2 sockets
Jun 19 08:18:44 dist-test-slave-d0b5 krb5kdc[25102](info): commencing operation
krb5kdc: starting...
W20260619 08:18:45.346896 2527 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/test-user.keytab.
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T08:18:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T08:18:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.119.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
Entry for principal kudu/127.2.119.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab.
I20260619 08:18:45.465780 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41949
--webserver_interface=127.2.119.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 08:18:45.546514 25115 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:45.546772 25115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:45.546801 25115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:45.546844 25115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:45.548157 25115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:45.548226 25115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:45.548251 25115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:45.548273 25115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:45.548307 25115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:45.549866 25115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.254.keytab
--principal=kudu/127.2.119.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.254:41949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.119.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:45.550143 25115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:45.550377 25115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:45.555533 25121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:45.555567 25115 server_base.cc:1034] running on GCE node
W20260619 08:18:45.555549 25123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:45.555544 25120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:45.556104 25115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:45.556397 25115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:45.557574 25115 hybrid_clock.cc:648] HybridClock initialized: now 1781857125557550 us; error 33 us; skew 500 ppm
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:45.566903 25115 init.cc:377] Logged in from keytab as kudu/127.2.119.254@KRBTEST.COM (short username kudu)
I20260619 08:18:45.567420 25115 webserver.cc:460] Webserver started at http://127.2.119.254:34541/ using document root <none> and password file <none>
I20260619 08:18:45.567685 25115 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:45.567760 25115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:45.567917 25115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:45.569097 25115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data/instance:
uuid: "6cda98d0b9524f2da9d6d181d71da44c"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7c5b8e5475a091e6f9ec2c9b1c1e3064"
server_key_iv: "02a3895d4be8882cd9830f76911afdb0"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.569522 25115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance:
uuid: "6cda98d0b9524f2da9d6d181d71da44c"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7c5b8e5475a091e6f9ec2c9b1c1e3064"
server_key_iv: "02a3895d4be8882cd9830f76911afdb0"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.571096 25115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 08:18:45.571935 25130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:45.572264 25115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 08:18:45.572391 25115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal
uuid: "6cda98d0b9524f2da9d6d181d71da44c"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7c5b8e5475a091e6f9ec2c9b1c1e3064"
server_key_iv: "02a3895d4be8882cd9830f76911afdb0"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.572482 25115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:45.579954 25115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:45.582093 25115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:45.582255 25115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:45.586086 25115 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.254:41949
I20260619 08:18:45.586117 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.254:41949 every 8 connection(s)
I20260619 08:18:45.586462 25115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/data/info.pb
I20260619 08:18:45.587483 25183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:45.589030 25183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.590731 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 25115
I20260619 08:18:45.590849 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-0/wal/instance
I20260619 08:18:45.591069 2527 external_mini_cluster.cc:1428] Setting key 5671a47e5f8abbccd3c606b136341a4e
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.119.253@KRBTEST.COM; defaulting to no policy
I20260619 08:18:45.601087 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.589741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.2.119.254:41949)
0619 08:18:45.589962 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.589966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.589990 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:18:45.590612 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.590615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.590645 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.590773 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.591396 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.592772 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.593996 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.594533 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.594662 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.596946 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.596957 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.596957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.596967 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.599139 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.599542 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.599546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.599546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.599660 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.600005 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.600006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.600006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.600369 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.600447 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.600672 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.600832 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260619 08:18:45.601096 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.589717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41949 (local address 127.0.0.1:39884)
0619 08:18:45.589890 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.589930 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.590421 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.590777 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.590779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.591111 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.591302 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.591306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.592863 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.592865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.593891 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.593895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.594634 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.594635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.594688 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.595247 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.595256 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.596840 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.599233 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.599239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.599241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.599438 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.599779 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.599780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.599780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.599920 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.600442 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.600448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.600514 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.600610 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.600758 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260619 08:18:45.601866 25183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.602319 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.602209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37414)
0619 08:18:45.602253 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.602299 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 08:18:45.602505 25134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.253@KRBTEST.COM" created.
I20260619 08:18:45.603006 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.602913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37420)
0619 08:18:45.602960 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.602988 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:18:45.603137 25183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:36303: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
Entry for principal kudu/127.2.119.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab.
I20260619 08:18:45.620151 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:36303
--webserver_interface=127.2.119.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:18:45.652390 25183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } attempt: 1
I20260619 08:18:45.652942 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.652789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37430)
0619 08:18:45.652868 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.652911 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 08:18:45.653386 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.653238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37432)
0619 08:18:45.653332 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.653362 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 08:18:45.653489 25183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:36303: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
W20260619 08:18:45.700515 25190 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:45.700733 25190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:45.700759 25190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:45.700780 25190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:45.701902 25190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:45.701942 25190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:45.701957 25190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:45.701969 25190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:45.701982 25190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:45.703383 25190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.253.keytab
--principal=kudu/127.2.119.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.253:36303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.119.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:45.703575 25190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:45.703748 25190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:45.708560 25195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:45.708624 25196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:45.708638 25198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:45.708981 25190 server_base.cc:1034] running on GCE node
I20260619 08:18:45.709221 25190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:45.709570 25190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:45.710749 25190 hybrid_clock.cc:648] HybridClock initialized: now 1781857125710725 us; error 35 us; skew 500 ppm
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:45.713500 25190 init.cc:377] Logged in from keytab as kudu/127.2.119.253@KRBTEST.COM (short username kudu)
I20260619 08:18:45.713994 25190 webserver.cc:460] Webserver started at http://127.2.119.253:46237/ using document root <none> and password file <none>
I20260619 08:18:45.714243 25190 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:45.714316 25190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:45.714473 25190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:45.715606 25190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data/instance:
uuid: "97d0bed0261244168203e06c1c6248f0"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7e8c1cd3c96c047dfaca41124adb79a7"
server_key_iv: "efc6562d13acd82b1afe9061c267d6e7"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.716001 25190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal/instance:
uuid: "97d0bed0261244168203e06c1c6248f0"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7e8c1cd3c96c047dfaca41124adb79a7"
server_key_iv: "efc6562d13acd82b1afe9061c267d6e7"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.717448 25190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:45.718202 25205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:45.718387 25190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:45.718487 25190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal
uuid: "97d0bed0261244168203e06c1c6248f0"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "7e8c1cd3c96c047dfaca41124adb79a7"
server_key_iv: "efc6562d13acd82b1afe9061c267d6e7"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.718571 25190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:45.721630 25183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } attempt: 2
I20260619 08:18:45.722050 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.721923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37440)
0619 08:18:45.721978 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.722020 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 08:18:45.722376 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.722291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37450)
0619 08:18:45.722333 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.722357 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 08:18:45.722476 25183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.253:36303: Network error: Client connection negotiation failed: client connection to 127.2.119.253:36303: connect: Connection refused (error 111)
I20260619 08:18:45.732355 25190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:45.734428 25190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:45.734540 25190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:45.738412 25190 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.253:36303
I20260619 08:18:45.738467 25257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.253:36303 every 8 connection(s)
I20260619 08:18:45.738821 25190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/data/info.pb
I20260619 08:18:45.739734 25258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:45.741367 25258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.745074 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 25190
I20260619 08:18:45.745186 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-1/wal/instance
I20260619 08:18:45.745451 2527 external_mini_cluster.cc:1428] Setting key 54a636f9e3462e57d0e06b3860f1538d
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:45.751525 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.742047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.2.119.254:41949)
0619 08:18:45.742091 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.742093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.742272 (+ 179us) server_negotiation.cc:407] Connection header received
0619 08:18:45.742697 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.742698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.742711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.742772 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.743377 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.743897 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.744875 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.745081 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.745164 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.747988 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.747990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.747990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.748004 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.749754 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.750257 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.750259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.750259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.750339 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.750710 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.750711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.750711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.750889 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.750994 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.751279 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.751386 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:45.751588 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.741997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41949 (local address 127.0.0.1:39900)
0619 08:18:45.742153 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.742193 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.742615 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.742839 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.742843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.743107 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.743293 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.743297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.743985 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.743986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.744773 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.744775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.745203 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.745205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.745395 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.745993 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.746002 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.747825 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.749884 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.749890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.749892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.750178 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.750433 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.750434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.750434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.750555 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.751005 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.751018 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.751139 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.751231 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.751369 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260619 08:18:45.752204 25258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.119.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.119.252@KRBTEST.COM" created.
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:45.760715 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.752521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.2.119.253:36303)
0619 08:18:45.752675 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.752681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.752705 (+ 24us) server_negotiation.cc:407] Connection header received
0619 08:18:45.752730 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.752732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.752747 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.752827 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.753088 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.754250 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.754985 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.755282 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.755346 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.757930 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.757938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.757938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.757946 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.759613 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.760007 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.760010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.760010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.760061 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.760246 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.760247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.760247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.760370 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.760430 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.760578 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.760644 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260619 08:18:45.760898 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.752455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37462)
0619 08:18:45.752501 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.752513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.752553 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.752844 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.752845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.752967 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.753015 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.753016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.754381 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.754381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.754914 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.754916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.755346 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.755347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.755425 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.755809 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.755813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.757846 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.759687 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.759688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.759689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.759923 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.760119 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.760120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.760120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.760188 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.760430 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.760431 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.760478 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.760555 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.760711 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":50}
I20260619 08:18:45.761626 25258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.761988 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.761851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53590)
0619 08:18:45.761898 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.761954 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 08:18:45.762131 25209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
I20260619 08:18:45.762564 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.762451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53600)
0619 08:18:45.762503 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.762535 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 08:18:45.762681 25258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:34513: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
Entry for principal kudu/127.2.119.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab.
I20260619 08:18:45.774389 2527 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskmYJIvA/build/release/bin/kudu
/tmp/dist-test-taskmYJIvA/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:34513
--webserver_interface=127.2.119.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.119.212:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 08:18:45.811920 25258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } attempt: 1
I20260619 08:18:45.812462 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.812289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53602)
0619 08:18:45.812359 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.812416 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 08:18:45.812865 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.812742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53616)
0619 08:18:45.812795 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.812832 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 08:18:45.813004 25258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:34513: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
I20260619 08:18:45.813688 25183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } attempt: 3
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:45.821257 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.814229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.2.119.253:36303)
0619 08:18:45.814282 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.814284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.814294 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:45.814322 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.814323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.814335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.814401 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.814694 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.815470 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.816198 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.816532 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.816604 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.818463 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.818465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.818465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.818472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.820033 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.820430 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.820432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.820432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.820489 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.820695 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.820696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.820696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.820842 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.820934 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.821093 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.821165 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:18:45.821257 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.814135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37474)
0619 08:18:45.814191 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.814203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.814253 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.814423 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.814424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.814567 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.814631 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.814633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.815554 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.815555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.816123 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.816125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.816610 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.816611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.816671 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.817129 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.817138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.818388 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.820117 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.820118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.820119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.820326 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.820560 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.820561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.820562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.820620 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.820929 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.820930 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.820983 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.821057 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.821159 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 08:18:45.821679 25183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.821995 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.821902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53622)
0619 08:18:45.821947 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.821973 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 08:18:45.822249 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.822183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53632)
0619 08:18:45.822208 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.822231 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 08:18:45.822331 25183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:34513: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
I20260619 08:18:45.853452 25183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } attempt: 1
I20260619 08:18:45.853945 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.853770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53644)
0619 08:18:45.853831 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.853915 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 08:18:45.854276 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.854185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53646)
0619 08:18:45.854228 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.854253 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 08:18:45.854367 25183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:34513: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
W20260619 08:18:45.860317 25265 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 08:18:45.860558 25265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 08:18:45.860594 25265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 08:18:45.860635 25265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 08:18:45.861797 25265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 08:18:45.861858 25265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 08:18:45.861881 25265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 08:18:45.861902 25265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 08:18:45.861929 25265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 08:18:45.863399 25265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.119.212:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.119.254:41949,127.2.119.253:36303,127.2.119.252:34513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/krb5kdc/kudu_127.2.119.252.keytab
--principal=kudu/127.2.119.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.119.252:34513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.119.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 06:52:04 UTC on ccycloud.quasar-kfejyx.root.comops.site
build id 4455157
I20260619 08:18:45.863662 25265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 08:18:45.863880 25265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 08:18:45.869074 25270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 08:18:45.869202 25271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:45.869310 25265 server_base.cc:1034] running on GCE node
W20260619 08:18:45.869096 25273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 08:18:45.869861 25265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 08:18:45.870204 25265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 08:18:45.871359 25265 hybrid_clock.cc:648] HybridClock initialized: now 1781857125871331 us; error 43 us; skew 500 ppm
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 08:18:45.874356 25265 init.cc:377] Logged in from keytab as kudu/127.2.119.252@KRBTEST.COM (short username kudu)
I20260619 08:18:45.874869 25265 webserver.cc:460] Webserver started at http://127.2.119.252:42953/ using document root <none> and password file <none>
I20260619 08:18:45.875130 25265 fs_manager.cc:359] Metadata directory not provided
I20260619 08:18:45.875180 25265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 08:18:45.875319 25265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 08:18:45.876600 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data/instance:
uuid: "bac9e6474ab545f282db41ad4a9c0ac5"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "ab890a12e67ce846fdf5001a40601fef"
server_key_iv: "795d70ebd3accbadd122e7b703a9408d"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.876986 25265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal/instance:
uuid: "bac9e6474ab545f282db41ad4a9c0ac5"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "ab890a12e67ce846fdf5001a40601fef"
server_key_iv: "795d70ebd3accbadd122e7b703a9408d"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.878528 25265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 08:18:45.879339 25280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 08:18:45.879514 25265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 08:18:45.879621 25265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data,/tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal
uuid: "bac9e6474ab545f282db41ad4a9c0ac5"
format_stamp: "Formatted at 2026-06-19 08:18:45 on dist-test-slave-d0b5"
server_key: "ab890a12e67ce846fdf5001a40601fef"
server_key_iv: "795d70ebd3accbadd122e7b703a9408d"
server_key_version: "encryptionkey@0"
I20260619 08:18:45.879698 25265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 08:18:45.881237 25258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } attempt: 2
I20260619 08:18:45.881739 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.881560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53658)
0619 08:18:45.881630 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.881662 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 08:18:45.882056 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.881965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53674)
0619 08:18:45.882005 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.882029 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 08:18:45.882139 25258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.119.252:34513: Network error: Client connection negotiation failed: client connection to 127.2.119.252:34513: connect: Connection refused (error 111)
I20260619 08:18:45.894296 25265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 08:18:45.896344 25265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 08:18:45.896495 25265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 08:18:45.900944 25265 rpc_server.cc:307] RPC server started. Bound to: 127.2.119.252:34513
I20260619 08:18:45.900977 25332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.119.252:34513 every 8 connection(s)
I20260619 08:18:45.901362 25265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/data/info.pb
I20260619 08:18:45.902343 25333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 08:18:45.903824 25333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } has no permanent_uuid. Determining permanent_uuid...
I20260619 08:18:45.909463 2527 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskmYJIvA/build/release/bin/kudu as pid 25265
I20260619 08:18:45.909578 2527 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskmYJIvA/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781856856097588-2527-0/minicluster-data/master-2/wal/instance
I20260619 08:18:45.909822 2527 external_mini_cluster.cc:1428] Setting key 81a32038cc56c26cd7df2a306a4a35c5
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
I20260619 08:18:45.913940 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.904506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.2.119.254:41949)
0619 08:18:45.904563 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.904565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.904764 (+ 199us) server_negotiation.cc:407] Connection header received
0619 08:18:45.905267 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.905268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.905279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.905469 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.906183 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.906658 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.907780 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.907969 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.908026 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.910596 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.910599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.910599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.910610 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.912335 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.912810 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.912811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.912812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.912876 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.913148 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.913149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.913149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.913259 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.913411 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.913751 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.913842 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 08:18:45.913966 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.904476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.254:41949 (local address 127.0.0.1:39902)
0619 08:18:45.904637 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.904683 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.905157 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.905557 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.905561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.905887 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.906095 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.906100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.906759 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.906759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.907658 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.907661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.908057 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.908058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.908176 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.908652 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.908661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.910506 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.912427 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.912433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.912435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.912731 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.912946 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.912946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.912947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.913081 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.913419 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.913425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.913584 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.913687 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.913811 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260619 08:18:45.914580 25333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:45.919907 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.910281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.2.119.252:34513)
0619 08:18:45.910450 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.910454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.910469 (+ 15us) server_negotiation.cc:407] Connection header received
0619 08:18:45.910565 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.910568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.910583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.910678 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.911408 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.912578 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.913583 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.914599 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.914659 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.916794 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.916802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.916802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.916810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.918745 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.919095 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.919097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.919098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.919158 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.919355 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.919356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.919356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.919493 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.919626 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.919718 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.919794 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260619 08:18:45.921844 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.914912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.2.119.253:36303)
0619 08:18:45.914965 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.914967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.915003 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:18:45.915086 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.915087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.915098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.915159 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.915438 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.915938 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.916713 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.917060 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.917138 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.919049 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.919050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.919050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.919058 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.920730 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.921060 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.921062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.921062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.921138 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.921363 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.921364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.921364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.921472 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.921543 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.921690 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.921758 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 08:18:45.921898 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.914843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.253:36303 (local address 127.0.0.1:37478)
0619 08:18:45.914962 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.914972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.915013 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.915173 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.915174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.915318 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.915362 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.915364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.916017 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.916019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.916630 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.916632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.917137 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.917138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.917211 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.917587 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.917591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.918956 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.920815 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.920815 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.920816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.920973 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.921216 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.921217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.921217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.921275 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.921534 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.921535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.921583 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.921664 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.921752 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260619 08:18:45.922194 25333 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.254@KRBTEST.COM
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.252@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:18:45.929167 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.920719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.2.119.254:41949)
0619 08:18:45.920766 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.920768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.920777 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:45.920809 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.920810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.920820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.920870 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.921511 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.922214 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.923135 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.923451 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.923517 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.926079 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.926080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.926080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.926087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.927969 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928341 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.928341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.928342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.928389 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928590 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.928591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.928591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.928739 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.928954 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.928995 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.929071 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 08:18:45.929464 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.922423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53692)
0619 08:18:45.922485 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.922494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.922532 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.922642 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.922643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.922777 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.922815 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.922817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.923620 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.923622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.924400 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.924402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.924774 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.924775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.924846 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.925244 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.925248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.926561 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.928263 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.928264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.928508 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.928704 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.928704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.928752 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.929070 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.929070 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.929118 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.929198 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.929356 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 08:18:45.929477 25183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } attempt: 2
I20260619 08:18:45.929737 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.922449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53692 (local address 127.2.119.252:34513)
0619 08:18:45.922479 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.922481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.922521 (+ 40us) server_negotiation.cc:407] Connection header received
0619 08:18:45.922585 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.922585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.922593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.922638 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.922874 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.923550 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.924492 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.924699 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.924759 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.926652 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.926653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.926654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.926661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.928189 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928589 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.928590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.928590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.928653 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.928868 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.928869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.928869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.928991 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.929063 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.929391 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.929460 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260619 08:18:45.931241 25333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: Bootstrap starting.
I20260619 08:18:45.932251 25333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:45.932621 25333 log.cc:824] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:45.933696 25333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: No bootstrap required, opened a new log
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.254@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.119.253@KRBTEST.COM
I20260619 08:18:45.936123 25333 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.936317 25333 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac9e6474ab545f282db41ad4a9c0ac5, State: Initialized, Role: FOLLOWER
I20260619 08:18:45.936416 25333 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.936785 25333 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:45.936882 25340 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.937393 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.929799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.2.119.252:34513)
0619 08:18:45.929918 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.929920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.929930 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:45.929953 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.929954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.929965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.930035 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.930334 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.931028 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.932066 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.932343 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.932427 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.934418 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.934419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.934420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.934427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.936089 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.936491 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.936493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.936493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.936556 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.936802 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.936803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.936804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.936957 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.937030 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.937195 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.937268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":129,"threads_started":1}
I20260619 08:18:45.937474 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.929680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53708)
0619 08:18:45.929718 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.929727 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.929759 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.930039 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.930040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.930201 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.930241 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.930243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.931117 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.931119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.931984 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.931987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.932434 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.932435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.932493 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.932986 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.932992 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.934327 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.936174 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.936175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.936176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.936417 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.936653 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.936654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.936655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.936723 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.937041 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.937042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.937092 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.937169 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.937284 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 08:18:45.936976 25340 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.938544 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.929758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.2.119.253:36303)
0619 08:18:45.929824 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.929826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.929972 (+ 146us) server_negotiation.cc:407] Connection header received
0619 08:18:45.930064 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.930065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.930075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.930104 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.930699 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.931300 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.931938 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.932233 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.932305 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.935382 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.935383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.935384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.935391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.937205 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.937636 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.937638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.937638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.937689 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.937869 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.937869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.937870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.937986 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.938131 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.938340 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.938409 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 08:18:45.939565 25183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: Bootstrap starting.
I20260619 08:18:45.939889 2527 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 08:18:45.940232 25183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:45.940532 25183 log.cc:824] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:45.941385 25183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: No bootstrap required, opened a new log
W20260619 08:18:45.941393 25351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:45.941437 25351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:45.940759 25157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6cda98d0b9524f2da9d6d181d71da44c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39914
I20260619 08:18:45.939283 25265 master_runner.cc:186] Error getting master registration for 127.2.119.254:41949: OK, instance_id { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" instance_seqno: 1781857125584941 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:18:45.942250 25265 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:18:45.942984 25183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.943118 25183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cda98d0b9524f2da9d6d181d71da44c, State: Initialized, Role: FOLLOWER
I20260619 08:18:45.943197 25183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.943516 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.943554 25183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:45.943562 25157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 08:18:45.943596 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.943658 25157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:45.944339 25157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.944538 25157 leader_election.cc:288] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [CANDIDATE]: Term 1 election: Requested vote from peers 97d0bed0261244168203e06c1c6248f0 (127.2.119.253:36303), bac9e6474ab545f282db41ad4a9c0ac5 (127.2.119.252:34513)
I20260619 08:18:45.944809 25307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6cda98d0b9524f2da9d6d181d71da44c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bac9e6474ab545f282db41ad4a9c0ac5"
I20260619 08:18:45.944924 25307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:45.945878 25307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cda98d0b9524f2da9d6d181d71da44c in term 1.
I20260619 08:18:45.946132 25134 leader_election.cc:302] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cda98d0b9524f2da9d6d181d71da44c, bac9e6474ab545f282db41ad4a9c0ac5; no voters:
I20260619 08:18:45.946249 25354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 08:18:45.946410 25354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [term 1 LEADER]: Becoming Leader. State: Replica: 6cda98d0b9524f2da9d6d181d71da44c, State: Running, Role: LEADER
I20260619 08:18:45.946504 25354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.946942 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cda98d0b9524f2da9d6d181d71da44c. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.947049 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: This master's current role is: LEADER
W20260619 08:18:45.947605 25373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 08:18:45.947705 25373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 08:18:45.948612 25115 master_runner.cc:186] Error getting master registration for 127.2.119.253:36303: OK, instance_id { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" instance_seqno: 1781857125737204 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 08:18:45.951938 25115 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 08:18:45.951996 25374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 08:18:45.952239 25374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 08:18:45.954495 25307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [term 1 FOLLOWER]: Refusing update from remote peer 6cda98d0b9524f2da9d6d181d71da44c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:45.955015 25354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:45.955431 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.946058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.2.119.252:34513)
0619 08:18:45.946271 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.946273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.946295 (+ 22us) server_negotiation.cc:407] Connection header received
0619 08:18:45.947346 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.947347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.947359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.947391 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.948238 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.948952 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.949908 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.950180 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.950218 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.952187 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.952188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.952189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.952197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.953916 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.954376 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.954377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.954377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.954478 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.954746 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.954747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.954748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.954905 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.954994 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.955212 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.955293 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260619 08:18:45.956413 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.945788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.2.119.253:36303)
0619 08:18:45.945840 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.945842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.946023 (+ 181us) server_negotiation.cc:407] Connection header received
0619 08:18:45.946168 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.946170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.946182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.946215 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.947485 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.948351 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.949821 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.950120 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.950160 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.952433 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.952435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.952435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.952443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.954155 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.955422 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.955423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.955424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.955492 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.955803 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.955804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.955804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.955939 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.956042 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.956211 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.956300 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:45.956570 25340 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cda98d0b9524f2da9d6d181d71da44c. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.956661 25340 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.957692 25376 mvcc.cc:205] Tried to move back new op lower bound from 7298486787906359296 to 7298486787877548032. Current Snapshot: MvccSnapshot[applied={T|T < 7298486787906359296}]
I20260619 08:18:45.957953 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.958039 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:45.958259 25340 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.958410 25340 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.962323 25374 catalog_manager.cc:1347] Generated new cluster ID: ef1e126b99574ced9ff2c903d132d194
I20260619 08:18:45.962384 25374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 08:18:45.968720 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.961601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39916 (local address 127.2.119.254:41949)
0619 08:18:45.961658 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.961660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.961766 (+ 106us) server_negotiation.cc:407] Connection header received
0619 08:18:45.961793 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.961794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.961804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.961861 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.962516 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.963236 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.964187 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.964389 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.964465 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.965864 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.965865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.965866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.965873 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.967497 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.967908 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.967909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.967909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.967959 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.968163 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.968164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.968164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.968293 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.968378 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.968545 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.968622 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 08:18:45.971989 25374 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 08:18:45.972882 25374 catalog_manager.cc:1504] Loading token signing keys...
I20260619 08:18:45.973268 25258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } attempt: 3
Jun 19 08:18:45 dist-test-slave-d0b5 krb5kdc[25102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781857125, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.119.253@KRBTEST.COM for kudu/127.2.119.252@KRBTEST.COM
I20260619 08:18:45.981462 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.973542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.119.252:34513 (local address 127.0.0.1:53720)
0619 08:18:45.973614 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 08:18:45.973625 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 08:18:45.973660 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 08:18:45.974090 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 08:18:45.974091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 08:18:45.974259 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 08:18:45.974312 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.974315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.975183 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.975184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.976006 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 08:18:45.976008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.976427 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.976428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 08:18:45.976480 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.976932 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 08:18:45.976937 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 08:18:45.978498 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 08:18:45.980330 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.980331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.980332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.980554 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.980765 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 08:18:45.980765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 08:18:45.980766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 08:18:45.980836 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 08:18:45.981123 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 08:18:45.981124 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 08:18:45.981181 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 08:18:45.981240 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 08:18:45.981362 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 08:18:45.981792 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:45.973795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.2.119.252:34513)
0619 08:18:45.973871 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:45.973872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:45.973878 (+ 6us) server_negotiation.cc:407] Connection header received
0619 08:18:45.973894 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:45.973894 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:45.973903 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:45.973946 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:45.974386 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.975120 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.976082 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:45.976373 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:45.976431 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:45.978590 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 08:18:45.978591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 08:18:45.978592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 08:18:45.978600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 08:18:45.980211 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.980629 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.980630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.980630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.980682 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 08:18:45.980900 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 08:18:45.980900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 08:18:45.980901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 08:18:45.981022 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 08:18:45.981079 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:45.981638 (+ 559us) server_negotiation.cc:299] Negotiation successful
0619 08:18:45.981708 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 08:18:45.983287 25258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: Bootstrap starting.
I20260619 08:18:45.984014 25258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: Neither blocks nor log segments found. Creating new log.
I20260619 08:18:45.984333 25258 log.cc:824] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: Log is configured to *not* fsync() on all Append() calls
I20260619 08:18:45.985255 25258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: No bootstrap required, opened a new log
I20260619 08:18:45.986222 25374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c: Generated new TSK 0
I20260619 08:18:45.986605 25374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 08:18:45.987722 25258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.987880 25258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d0bed0261244168203e06c1c6248f0, State: Initialized, Role: FOLLOWER
I20260619 08:18:45.988047 25258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } }
I20260619 08:18:45.988389 25383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.988658 25383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.988502 25232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6cda98d0b9524f2da9d6d181d71da44c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "97d0bed0261244168203e06c1c6248f0"
I20260619 08:18:45.988478 25231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 08:18:45.988415 25258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 08:18:45.989869 25231 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [term 1 FOLLOWER]: Refusing update from remote peer 6cda98d0b9524f2da9d6d181d71da44c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 08:18:45.990221 25368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [LEADER]: Connected to new peer: Peer: permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 08:18:45.991657 25232 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cda98d0b9524f2da9d6d181d71da44c in term 1.
I20260619 08:18:45.995414 25397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cda98d0b9524f2da9d6d181d71da44c. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.999099 25397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:45.999155 25397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:45.999212 25397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 08:18:46.002004 25394 catalog_manager.cc:1259] Loaded cluster ID: ef1e126b99574ced9ff2c903d132d194
I20260619 08:18:46.002060 25394 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: loading cluster ID for follower catalog manager: success
I20260619 08:18:46.003351 25394 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: acquiring CA information for follower catalog manager: success
I20260619 08:18:46.003715 25394 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97d0bed0261244168203e06c1c6248f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 08:18:46.016755 25262 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.014960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.2.119.253:36303)
0619 08:18:46.015010 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.015013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.016047 (+ 1034us) server_negotiation.cc:407] Connection header received
0619 08:18:46.016166 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.016167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.016179 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.016213 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.016682 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37512: BlockingRecv error: recv got EOF from 127.0.0.1:37512 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 08:18:46.017889 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.015364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.2.119.252:34513)
0619 08:18:46.015405 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.015407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.016993 (+ 1586us) server_negotiation.cc:407] Connection header received
0619 08:18:46.017256 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.017258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.017341 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.017376 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.017824 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53728: BlockingRecv error: recv got EOF from 127.0.0.1:53728 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:46.018546 25186 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.015742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.2.119.254:41949)
0619 08:18:46.015794 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.015797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.017201 (+ 1404us) server_negotiation.cc:407] Connection header received
0619 08:18:46.018099 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.018101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.018113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.018144 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.018464 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39922: BlockingRecv error: recv got EOF from 127.0.0.1:39922 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:46.029213 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.022009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.2.119.253:36303)
0619 08:18:46.022063 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.022066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.025491 (+ 3425us) server_negotiation.cc:407] Connection header received
0619 08:18:46.025615 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.025616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.025663 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.025713 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.025956 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.026832 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.028131 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.028468 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.028514 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.028756 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.028914 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.028972 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.029101 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.029124 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:46.033033 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.032116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.2.119.252:34513)
0619 08:18:46.032162 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.032165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.032394 (+ 229us) server_negotiation.cc:407] Connection header received
0619 08:18:46.032524 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.032525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.032545 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.032577 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.032969 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53742: BlockingRecv error: recv got EOF from 127.0.0.1:53742 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:46.039026 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.035333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.2.119.253:36303)
0619 08:18:46.035369 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.035371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.035624 (+ 253us) server_negotiation.cc:407] Connection header received
0619 08:18:46.035747 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.035748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.035758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.035800 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.035983 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.036789 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.038140 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.038472 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.038515 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.038720 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.038756 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.038798 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.038912 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.038932 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:18:46.042239 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.041373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53744 (local address 127.2.119.252:34513)
0619 08:18:46.041413 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.041415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.041657 (+ 242us) server_negotiation.cc:407] Connection header received
0619 08:18:46.041760 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.041761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.041773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.041818 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.042182 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53744: BlockingRecv error: recv got EOF from 127.0.0.1:53744 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 08:18:46.052806 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.045393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.2.119.253:36303)
0619 08:18:46.045441 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.045443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.049492 (+ 4049us) server_negotiation.cc:407] Connection header received
0619 08:18:46.049604 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.049605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.049616 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.049647 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.049845 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.050671 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.051898 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.052243 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.052286 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.052508 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.052545 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.052574 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.052685 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.052706 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 08:18:46.056193 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.055319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.2.119.252:34513)
0619 08:18:46.055361 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.055363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.055592 (+ 229us) server_negotiation.cc:407] Connection header received
0619 08:18:46.055715 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.055716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.055727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.055760 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.056134 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53752: BlockingRecv error: recv got EOF from 127.0.0.1:53752 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:46.066386 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.061420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.2.119.253:36303)
0619 08:18:46.061461 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.061463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.061767 (+ 304us) server_negotiation.cc:407] Connection header received
0619 08:18:46.061985 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.061987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.061998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.062031 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.063350 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.064196 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.065476 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.065817 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.065860 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.066102 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.066138 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.066168 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.066281 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.066301 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:46.069725 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.068816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.2.119.252:34513)
0619 08:18:46.068859 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.068861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.069128 (+ 267us) server_negotiation.cc:407] Connection header received
0619 08:18:46.069234 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.069235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.069247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.069278 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.069665 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53754: BlockingRecv error: recv got EOF from 127.0.0.1:53754 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 08:18:46.082329 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.078919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.2.119.253:36303)
0619 08:18:46.078960 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.078962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.079245 (+ 283us) server_negotiation.cc:407] Connection header received
0619 08:18:46.079350 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.079351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.079361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.079390 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.079565 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.080329 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.081511 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.081814 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.081853 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.082047 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.082083 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.082110 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.082229 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.082248 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 08:18:46.085669 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.084833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.2.119.252:34513)
0619 08:18:46.084876 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.084878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.085084 (+ 206us) server_negotiation.cc:407] Connection header received
0619 08:18:46.085190 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.085191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.085202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.085231 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.085612 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53764: BlockingRecv error: recv got EOF from 127.0.0.1:53764 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 08:18:46.106587 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.103027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.2.119.253:36303)
0619 08:18:46.103071 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.103073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.103382 (+ 309us) server_negotiation.cc:407] Connection header received
0619 08:18:46.103490 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.103492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.103502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.103532 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.103727 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.104527 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.105721 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.106024 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.106072 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.106284 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.106319 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.106347 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.106489 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.106509 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:46.109869 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.108956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.2.119.252:34513)
0619 08:18:46.108998 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.108999 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.109244 (+ 245us) server_negotiation.cc:407] Connection header received
0619 08:18:46.109398 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.109399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.109411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.109441 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.109814 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53780: BlockingRecv error: recv got EOF from 127.0.0.1:53780 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:46.146842 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.143166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.2.119.253:36303)
0619 08:18:46.143212 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.143214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.143616 (+ 402us) server_negotiation.cc:407] Connection header received
0619 08:18:46.143761 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.143763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.143774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.143802 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.144016 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.144802 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.146002 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.146290 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.146334 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.146535 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.146569 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.146595 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.146745 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.146766 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 08:18:46.150741 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.149891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.2.119.252:34513)
0619 08:18:46.149936 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.149938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.150147 (+ 209us) server_negotiation.cc:407] Connection header received
0619 08:18:46.150248 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.150249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.150260 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.150290 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.150672 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53794: BlockingRecv error: recv got EOF from 127.0.0.1:53794 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 08:18:46.218883 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.216106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.2.119.253:36303)
0619 08:18:46.216166 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.216168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.216299 (+ 131us) server_negotiation.cc:407] Connection header received
0619 08:18:46.216351 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.216351 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.216361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.216413 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.216563 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.217117 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.218265 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.218453 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.218508 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.218625 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.218657 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.218696 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.218761 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.218777 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 08:18:46.221379 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.220627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.2.119.252:34513)
0619 08:18:46.220692 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.220694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.220711 (+ 17us) server_negotiation.cc:407] Connection header received
0619 08:18:46.220801 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.220802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.220826 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.220891 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.221267 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53804: BlockingRecv error: recv got EOF from 127.0.0.1:53804 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 08:18:46.352775 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.350417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.2.119.253:36303)
0619 08:18:46.350515 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.350517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.350589 (+ 72us) server_negotiation.cc:407] Connection header received
0619 08:18:46.350649 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.350650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.350660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.350703 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.350819 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.351324 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.352098 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.352296 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.352354 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.352516 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.352548 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.352605 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.352649 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.352668 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 08:18:46.355087 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.354468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53810 (local address 127.2.119.252:34513)
0619 08:18:46.354540 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.354542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.354578 (+ 36us) server_negotiation.cc:407] Connection header received
0619 08:18:46.354656 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.354656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.354667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.354735 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.355023 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53810: BlockingRecv error: recv got EOF from 127.0.0.1:53810 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 08:18:46.429243 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6cda98d0b9524f2da9d6d181d71da44c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cda98d0b9524f2da9d6d181d71da44c" member_type: VOTER last_known_addr { host: "127.2.119.254" port: 41949 } } peers { permanent_uuid: "97d0bed0261244168203e06c1c6248f0" member_type: VOTER last_known_addr { host: "127.2.119.253" port: 36303 } } peers { permanent_uuid: "bac9e6474ab545f282db41ad4a9c0ac5" member_type: VOTER last_known_addr { host: "127.2.119.252" port: 34513 } } }
I20260619 08:18:46.429469 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cda98d0b9524f2da9d6d181d71da44c [sys.catalog]: This master's current role is: LEADER
I20260619 08:18:46.614705 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:46.612373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.2.119.253:36303)
0619 08:18:46.612439 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.612441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.612452 (+ 11us) server_negotiation.cc:407] Connection header received
0619 08:18:46.612509 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.612510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.612518 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.612577 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:46.612786 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.613284 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.614034 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:46.614214 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:46.614271 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:46.614448 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:46.614475 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:46.614513 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:46.614577 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 08:18:46.614595 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 08:18:46.617147 25337 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 08:18:46.616599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.2.119.252:34513)
0619 08:18:46.616647 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:46.616649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:46.616658 (+ 9us) server_negotiation.cc:407] Connection header received
0619 08:18:46.616728 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:46.616729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:46.616745 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:46.616783 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0619 08:18:46.617095 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53816: BlockingRecv error: recv got EOF from 127.0.0.1:53816 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 08:18:46.941877 25351 catalog_manager.cc:1259] Loaded cluster ID: ef1e126b99574ced9ff2c903d132d194
I20260619 08:18:46.941973 25351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: loading cluster ID for follower catalog manager: success
I20260619 08:18:46.942782 25351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: acquiring CA information for follower catalog manager: success
I20260619 08:18:46.943149 25351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bac9e6474ab545f282db41ad4a9c0ac5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 08:18:47.132974 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:47.130351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.2.119.253:36303)
0619 08:18:47.130495 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:47.130497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:47.130591 (+ 94us) server_negotiation.cc:407] Connection header received
0619 08:18:47.130688 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:47.130689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:47.130702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:47.130780 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:47.130964 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.131472 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.132243 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.132452 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.132534 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:47.132676 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:47.132717 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:47.132774 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:47.132851 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 08:18:47.132874 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":54,"threads_started":1}
I20260619 08:18:47.137116 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:47.134649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.2.119.252:34513)
0619 08:18:47.134769 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:47.134771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:47.134781 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:47.134807 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:47.134808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:47.134880 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:47.134961 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:47.135074 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.135596 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.136329 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.136566 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.136648 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:47.136803 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:47.136942 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:47.137003 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:47.137040 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 08:18:47.137056 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
I20260619 08:18:47.141479 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:47.138786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.2.119.253:36303)
0619 08:18:47.138853 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:47.138856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:47.138866 (+ 10us) server_negotiation.cc:407] Connection header received
0619 08:18:47.138929 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:47.138930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:47.138939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:47.138998 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:47.139146 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.139888 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.140708 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.140933 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.140994 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:47.141155 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:47.141188 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:47.141263 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:47.141367 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 08:18:47.141388 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 08:18:47.141919 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:47.138875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.2.119.252:34513)
0619 08:18:47.138933 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:47.138936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:47.139197 (+ 261us) server_negotiation.cc:407] Connection header received
0619 08:18:47.139220 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:47.139220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:47.139229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:47.139272 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:47.139447 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.140118 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.141093 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.141391 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.141448 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:47.141604 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:47.141629 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:47.141674 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:47.141806 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 08:18:47.141844 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 08:18:47.142035 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 08:18:47.138749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.2.119.254:41949)
0619 08:18:47.138947 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 08:18:47.138949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 08:18:47.138997 (+ 48us) server_negotiation.cc:407] Connection header received
0619 08:18:47.139072 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 08:18:47.139073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 08:18:47.139129 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 08:18:47.139218 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 08:18:47.139313 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.139856 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.140907 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 08:18:47.141208 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 08:18:47.141459 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 08:18:47.141525 (+ 66us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 08:18:47.141629 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 08:18:47.141906 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 08:18:47.141925 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 08:18:47.141949 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":65,"threads_started":1}
I20260619 08:18:47.143659 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 25115
I20260619 08:18:47.148721 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 25190
I20260619 08:18:47.153651 2527 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskmYJIvA/build/release/bin/kudu with pid 25265
2026-06-19T08:18:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2860 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5652 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (271063 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST